Deprecated document_type in Logstash 6

Elastic Stack Logstash
1

Hello,
See in the breaking Changes (Breaking changes | Logstash Reference [8.11] | Elastic) that:

"The default document_type has changed from logs to doc for consistency with Beats"

What does this mean for me?

I am using this output like this:
elasticsearch {
hosts => "elasticsearchserver:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}

Does this just mean that the type property will be removed from documents?
If i dont use that, can i remove that row?

2

Have a look at Help please.. breaking changes?

3

That is Another thread i created, the problem was that i had not uploaded a new beat template into elasticsearch. Now i have uploaded the template and i dont see any errors, but i see this in my logstash log:

[2017-11-17T11:11:07,182][WARN ][logstash.outputs.elasticsearch] You are using a deprecated config setting "document_type" set in elasticsearch. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Document types are being deprecated in Elasticsearch 6.0, and removed entirely in 7.0. You should avoid this feature If you have any questions about this, please visit the logstash channel on freenode irc. {:name=>"document_type", :plugin=><LogStash::Outputs::Elasticsearch hosts=>"vascmdbtest01:9200", manage_template=>"false", index=>"%{[@metadata][beat]}-%{[@metadata][version]}-dc-%{+YYYY.MM.dd}", document_type=>"%{[@metadata][type]}", id=>"6f734d37a3f67008eddcd9167d4a741ea61831ed43f9528428cbb1c915e8ebaf">}

if i remove the following line from the logstash output block to elasticsearch:
document_type => "%{[@metadata][type]}"

I get these warnings:
[2017-11-17T13:18:20,163][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-6.0.0-2017.11.17", :_type=>"wineventlog", :_routing=>nil}, #LogStash::Event:0x2b15d6c6], :response=>{"index"=>{"_index"=>"winlogbeat-6.0.0-2017.11.17", "_type"=>"wineventlog", "_id"=>"0OnqyV8BA5Maqd7NWlSX", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [winlogbeat-6.0.0-2017.11.17] as the final mapping would have more than 1 type: [doc, wineventlog]"}}}}

Why? How do i get away from using document_type?

1 Like
4

Ok, let me close this one in favour of that given there is more activity there :slight_smile: