Logstash Filter meter is writing output to tag_field variable instead of its value


(Kapil514) #1

Below is my logstash configuration.

cat /etc/logstash/conf.d/logstash.conf
input {
beats {
port => "5044"
}
}
filter {
mutate {
gsub => ["host", '.', '_']
}
if [type] == "log" {
metrics {
meter => ["logstash.host.logaggregator"]
}
}
}
output {
file {
path => "/data/logstash/%{[fields][app_id]}.out"
}
graphite{
host => "graphite_server.example.com"
port => "2003"
include_metrics => ["logstash.*"]
fields_are_metrics => true
}
stdout {
codec => rubydebug
}
}

With above configuration all things are working fine but when the logs are getting written to logfile the output from file beat is written to %{[fields][app_id]}.out tag value (value of %{[fields][app_id]} is app1,app2 and so on) and there is one more file with name "%{[fields][app_id]}.out" which is having the value of rate_1m,rate_5m,count I am not sure why the meter output is not written to the same file of tag value instead of tag variable.

filebeat lines process to file : app1.out ,app2.out and so on 
meter rate output of all apps : %{[fields][app_id]}.out 

What configuration mistake I am doing . Thanks in advance for your help on this.


(Kapil514) #2

This has resolved I did created a tag inside meter and while writing to file I wrote check to not to write lines which don't have meter tag and for graphite I checked for tag and wrote the metric to graphite.

if [type] == "log" {
metrics {
meter => ["logstash.host.logaggregator"]
add_tag => "graphite_metric"
}
}
}
output {
if "graphite_metric" not in [tags] {
file {
path => "/data/logstash/%{[fields][app_id]}.out"
} }
if "graphite_metric" in [tags] {
graphite{
host => "graphtie_host"
port => "2003"
include_metrics => ["logstash.*"]
fields_are_metrics => true
} }


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.