Do I need to add new field manually in Elasticsearch?

Elastic Stack Kibana
1

Hi!
I'm using an ELK setup where I use Grok to filter data inside Logstash. I have defined my Grok filters inside of 02-filebeats-input.conf file like below and I wonder if I in some Elasticsearch way have to manually add the new fields I specified in the Grok filter (queue and windowsize for instance).

The reason why I'm asking is because I cannot see the fields windowsize or queue in Kibana's discover view -> available fields. I can only see them if all my shown results contain this field.

filter {
  grok {

     match => { "message" => "%{TIMESTAMP_ISO8601} %{LOGLEVEL} SomeManager: <ChangeQueue gate: %{GREEDYDATA:**queue**}> window size (in|de)creased to %{INT:**windowsize**}" }

     match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:log_level} %{USERNAME:service_name}: %{GREEDYDATA:contents}" }

  }
}
2

Elasticsearch will automatically add the new fields to the index mapping. However, the index pattern needs to be refreshed in Kibana in order for the new fields to show up in Kibana. Go to Management -> Index Patterns and select your Index Pattern. Then click the refresh field list button in the upper right corner.

3

Thanks!!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.