Logstash doesn't automatically collect all Zeek fields without grok pattern

Elastic Stack Logstash
#1

Hi,

Is there a setting I need to provide in order to enable the automatically collection of all the Zeek's log fields? I can collect the fields message only through a grok filter.

My assumption is that logstash is smart enough to collect all the fields automatically from all the Zeek log types. Is this right?

PS I don't have any plugin installed or grok pattern provided.
My pipeline is zeek-filebeat-kafka-logstash

Thank you in advance!

Thank you!

#2

Automatic field detection is only possible with input plugins in Logstash or Beats . But logstash doesn't have a zeek log plugin .

File Beat have a zeek module . If you are using this , Filebeat will detect zeek fields and create default dashboard also

Detecting Zeek Fields

#3

@Automation_Scripts if you have setup Zeek to log in json format, you can easily extract all of the fields in Logstash using the json filter. For example:

json {
  skip_on_invalid_json => true
  source => "message"
  target => "zeek"
}
#4

Thank you! Seems that my zeek was logging TSV and not Json. Thank your for your hint. Now I have to ser why filebeat doesnt do it’s enrichment of the data ==> ECS i.e I hve no event.dataset etc.