My goal is to monitor the health of a node so my F5 LB can determine which nodes in the cluster are healthy before sending logs to them. I'm trying to sett up the heartbeat according to F5 High Speed Logging with Elastic Stack, but there are a few minor issues likely due to the article being based on an older version. I did my best to get it working on 6.5 but I'm getting the following error:
Dec 23 04:42:20 elk logstash[3077]: [2018-12-23T04:42:20,566][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"elk", :_index=>"heartbeats", :_type=>"doc", :_routing=>nil}, #LogStash::Event:0x5e3db28f], :response=>{"index"=>{"_index"=>"heartbeats", "_type"=>"doc", "_id"=>"elk", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [heartbeats] as the final mapping would have more than 1 type: [heartbeat, doc]"}}}}
Here is my conf
input {
tcp {
port => 9900
type => syslog
}
udp {
port => 9900
type => syslog
}
heartbeat {
interval => 5
type => heartbeat
}
}output {
if [type] == "heartbeat" {
elasticsearch {
index => "heartbeats"
document_id => "%{host}"
}
}
else {
elasticsearch { hosts => ["192.168.8.19:9200"] }
}
}
Here is my index mapping.
{
"mapping": {
"heartbeat": {
"properties": {
"host": {
"type": "keyword"
}
}
}
}
}
Before I created the heartbeats index, my conf file did not have the conditional in the output stanza and the heartbeat messages were delivered to the default index. Is it possible that logstash isn't actually setting the type to 'heartbeat'?