I have previously commented on another topic with a similar issue, but as that was beats related, and fixed by updates there, and mine is not, i a, creating this topic.
I am getting errors due to multiple mapping types, as per the 6.0 breaking changes, but i don't know why i have multiple mapping types.
The data is delivered by a powershell script which gathers performance counter data and sends it to logstash. My logstash configuration looks like this:
input {
tcp {
port => 5560
codec => "json"
type => ["performancecounter"]
}
}
filter {
if [type] == "performancecounter" {
grok {
match => { "CounterPath" => [ "\\\\%{DATA:hostname}\\%{DATA:countergroup}\(%{DATA:hostname}\\private\$\\%{DATA:service}\)\\%{GREEDYDATA:Counter}",
"\\\\%{DATA:hostname}\\%{DATA:countergroup}\\%{GREEDYDATA:Counter}" ] }
}
date {
"match" => [ "MeasureTime", "yyyy-MM-dd HH:mm:ss,SSS" ]
target => "@timestamp"
}
}
}
output {
if [type] == "performancecounter" {
elasticsearch {
hosts => ["172.22.22.195:9200"]
index => "performance-%{+YYYY.MM.dd}"
}
}
}
With this i get the following error:
[2017-11-17T10:50:46,492][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"performance-2017.11.17", :_type=>"performancecounter", :_routing=>nil}, #<LogStash::Event:0x126f6108>], :response=>{"index"=>{"_index"=>"performance-2017.11.17", "_type"=>"performancecounter", "_id"=>"9KhjyV8BDxseqbvrQakY", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [performance-2017.11.17] as the final mapping would have more than 1 type: [performancecounter, json]"}}}}
I have tried to work around it by removing the type and using add_field to create a custom field, then check for that instead. But then i end up with this error:
[2017-11-17T10:58:27,378][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"performance-2017.11.17", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x345fcc96>], :response=>{"index"=>{"_index"=>"performance-2017.11.17", "_type"=>"doc", "_id"=>"dqhqyV8BDxseqbvrRboa", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [performance-2017.11.17] as the final mapping would have more than 1 type: [json, doc]"}}}}
I have other powershell scripts gathering performance counter data and delivering them to very similar configurations and i don't get an error. Where is that JSON type comming from? Is there a way to strip this in the filter or output, removing one value or forcing a specific value? I have tried using mutate in the filter to replace the type field with a new (the same) value but it doesn't change anything.
