Mapper_parsing_exception", "reason"=>"[include_in_all - is not allowed for indices created on or after version 6.0.0

Hi,

I get the following error:

[2017-11-29T12:43:23,143][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=> "logstash-2017.11.29", :_type=>"syslog", :_routing=>nil}, #<LogStash::Event:0x60 16b71a>], :response=>{"index"=>{"_index"=>"logstash-2017.11.29", "_type"=>"syslo g", "_id"=>nil, "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "re ason"=>"Failed to parse mapping [default]: [include_in_all] is not allowed for indices created on or after version 6.0.0 as [all] is deprecated. As a replace ment, you can use an [copy_to] on mapping fields to create your own catch all fi eld.", "caused_by"=>{"type"=>"mapper_parsing_exception", "reason"=>"[include_in all] is not allowed for indices created on or after version 6.0.0 as [_all] is d eprecated. As a replacement, you can use an [copy_to] on mapping fields to creat e your own catch all field."}}}}}

What does it excatly mean. In 5.6 everything worked fine. Is there problem in the conf.d folder and files?
inputOutput.conf:

syslog {
type => "syslog"
port => 5514
}
udp {
type => "syslog"
port => 5140
}
tcp {
type => "syslog"
port => 5140
}
beats {
port => 5044
}
udp {
type => "syslog"
port => 5515

}
}

output {
if [@metadata][beat] {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
} else {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "logstash-%{+YYYY.MM.dd}"
}
}
}

I can provide all other files of course if needed.

Thanks,

In Elasticsearch 6.0 the _all field is disabled/deprecated as outlined in the section in the docs about breaking mapping changes.

I read that in this moment....now I have to understand what it excactly means.

I have the following files:

inputOutput.conf -> https://pastebin.com/w0kN811p
10-syslog.conf -> https://pastebin.com/7wbLuNVU
11-pfsense.conf -> https://pastebin.com/8fttPiiq

How can I now check that if there is such an all_field?

It seems you are using include_in_all in your mapping template, which is causing this.

1 Like

You're right.

My logstash template looks like that. https://pastebin.com/sgHbcAkc

I have deleted it now. But now I have to create a new one. What parameters would you set?
I have "_all": { and "include_in_all": false but I don't know why. I have never set that as far as i know. I updated from 5.2 to 5.4 and from 5.4 to 5.6 and now 6.

You probably should upgrade to Logstash 6.0 and use the template that comes with it, as it looks like you are using the default Logstash template.

Thanks this worked.

I have used: https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/master/lib/logstash/outputs/elasticsearch/elasticsearch-template-es6x.json

I hope this was the correct one?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.