Filebeat fields are not working

Elastic Stack Logstash
1

Here is my filebeat config:

filebeat.prospectors:
- type: log
  paths:
    - C:/inetpub/app01/logs/IIS/*/*.log
  fields:
    app_group: iis
    app_id: app01

- type: log
  paths:
    - C:/inetpub/app02/logs/IIS/*/*.log
  fields:
    app_group: iis
    app_id: app02

- type: log
  paths:
    - C:/inetpub/app03/logs/IIS/*/*.log
  fields:
    app_group: iis
    app_id: app03

output:
  logstash:
    hosts: ["myserver:7777"]

And here is logstash:

input {
  beats {
    port => 7777
  }
}

filter {
  if [fields][app_group] == "iis" {
    .... do my parsing ...
  }
}

output {
  if [fields][app_group] == "iis" {
    elasticsearch {
      hosts => "mycluster"
      index => "logstash-iis-%{[fields][app_id]}-%{+YYYY-MM-dd}"
    }
  } else {
    elasticsearch {
      hosts => "mycluster"
      index => "uncategorized"
    }
  }
}

They are all getting pushed into my "uncategorized" index which means the fields aren't getting applied or if [fields][app_group] is wrong or something. I can't find any example with setting multiple fields and having logstash access them

2

Both configurations look OK to me. You don't have fields_under_root set in filebeat, right?

I suggest you look at the events in elasticsearch and make sure they really do have [fields][app_group] set.

3

yeah i don't have fields_under_root set. Should i be able to see the "fields" for the events in kibana?

4

Yes. Look at this post for an example.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.