Beat.name not populated with logstash output

[christiaanvanaken]

Data from Metricbeat is send to my central Logstash server and from there the output goes to Elasticsearch. If I do this, the field beat.name is always empty. If I change the Metricbeat configuration to send it to Elasticsearch directly, the field beat.name is correctly filled in with the hostname. I followed the manual for installing and configuring Metricbeat.

Please help!

[ruflin]

Please share your beat version, beat config, LS config, LS version.

[christiaanvanaken]

metricbeat version 5.4.0
logstash 5.4.0

file /etc/metricbeat/metricbeat.yml:

#----------------------------- Logstash output --------------------------------
output.logstash:

The Logstash hosts

hosts: ["monserver:5044"]

Optional SSL. By default is off.

List of root certificates for HTTPS server verifications

ssl.certificate_authorities: ["/etc/metricbeat/certificate.crt"]

Certificate for SSL client authentication

#ssl.certificate: "/etc/pki/client/cert.pem"

Client Certificate Key

#ssl.key: "/etc/pki/client/cert.key"

file /etc/logstash/conf.d/02-metricbeat.conf:

input {
beats {
port => 5044
codec => "json_lines"
ssl => true
ssl_certificate => "/etc/logstash/certificate.crt"
ssl_key => "/etc/logstash/certificate.key"
}
}

output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}

[ruflin]

Why do you use the json codec?

[christiaanvanaken]

Well it does not matter if I put it in there or not. Either way it does give me the beat.name.

[ruflin]

Can you enable the ruby debug output on LS to see if the events you receive that the beat.name set in it?

[christiaanvanaken]

output received {"event"=>{"@timestamp"=>2017-05-24T12:48:18.417Z, "system"=>{"process"=>{"memory"=>{"rss"=>{"pct"=>0.0009, "bytes"=>3497984}, "size"=>224391168, "share"=>2859008}, "cmdline"=>"/usr/local/psa/admin/bin/modules/watchdog/monit -Ic /usr/local/psa/etc/modules/watchdog/monitrc", "pgid"=>824, "name"=>"monit", "cpu"=>{"start_time"=>"2017-05-02T22:03:36.000Z", "total"=>{"pct"=>0.0}}, "pid"=>824, "state"=>"sleeping", "fd"=>{"limit"=>{"hard"=>4096, "soft"=>1024}, "open"=>5}, "ppid"=>1, "username"=>"root"}}, "host"=>"", "metricset"=>{"rtt"=>55135, "module"=>"system", "name"=>"process"}, "GeoLocation"=>{}}}

I suppressed the real hostname, but it looks like the beat stuff is not coming to LS.

[christiaanvanaken]

Problem solved, my mistake. I have more then one inpuf defined so I needed to add some tags to make sure the right filters and outputs are used.