Filebeat & Logstash configuration 6.5.4

Elastic Stack Logstash
1

Hi There,
I have installed ELK & filebeat version 6.5.4. Kibana & ES configuration is good.
From a server I want to send logs (.csv format) from 3 different location but facing problem in configuration somewhere i think.
Below is my configuration please check -

Filebeat 6.5.4 -

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /data/maalog/maa-coupon.log
  fields:
    log_type: maa-coupon
  fields_under_root: true

- type: log
  enabled: true
  paths:
    - /data/maalog/maa-tran.log
  fields:
    log_type: maa-tran
#   input_type: log
  fields_under_root: true

output.logstash:
  # The Logstash hosts
  hosts: ["10.5.100.100:5000"]

Logstash 6.5.4 -

input {
  beats {
    port => "5000"
  }
}

filter{

        if "maa-coupon" in [fields][log_type] {
        csv{
                separator => ","
                columns => ["lob","userId","mobileNo","circleId"]
           }
}
}

output {
  if "maa-coupon" in [fields][log_type] {
     elasticsearch {
        hosts => ["10.5.100.100:9200"]
        index => "test-coupon-log-%{+YYYY.MM.dd}"
      #workers => 4
     }
  }
}

when I don't use filter then logs were sent to ES & visible in Kibana but in message section only. it'll not appear in separate fields as there is no grok/filter.

What wrong in my configuration & how can I achieve my goal? Please suggest.
Thanks.

2

If you have fields_under_root set true then you can refer to just [log_type], not [fields][log_type]

3

Thanks Badger. I got you & its working now.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.