Now that support for type has been removed in filebeat 6, how to add filters in logstash config?

From what I read, filebeat does not honor document_type anymore (from version 6 onwards) and all _type gets hardcoded to doc.

I read some people getting around this by adding a field called document_type. Something like this in filebeat yml

filebeat.prospectors:

    - type: log

      # Change to true to enable this prospector configuration.
      enabled: true

      # Paths that should be crawled and fetched. Glob based paths.
      paths:
        - XXX\LogFiles\W3SVC1\*.log
      fields:
        document_type: iis_log

I verified that this ends up in elasticsearch. But now my question is how do i add a filter on this in logstash? For example in the previous way, i would had done something like this in logstash config

filter {
   

    if [type] == "iis_log" {
       

        grok {
            match => { xxx}
        } 

    }
}

What do I replace the if [type] == "iis_log" in the config, if its in a field now?

1 Like

There's no difference except the field name. type was also just a field. Use

if [fields][document_type] == "iis_log" {

or whatever it's called.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.