Filebeat 6.2.1 and logstash 6.2.1 and doc type

Hi,

I recently updated ELK from 5.4 to 6.2.1

My upgrade went reasonably well and i modified some custom templates on the elasticsearch side in relation to SRX data and geoip. That all works.

My question is really in regard to filebeat 6.2.1 and logstash 6.2.1
I currently have 2 x filebeat templates in ES. Those are the "version" : "5.4.2" and "filebeat-6.2.1" : {
"order" : 1,
"index_patterns" : [
"filebeat-6.2.1-*"
],

In fear of boring you i wont post them here, the templates, which are as best i can see, the templates provided by default and installation of filebeats.

Those templates have a index-patterns and an order. Does 1 of these "the older" require deletion?

I read about the 6.0.0 and type being doc for logstash to es and compatibility with beats.

I have the document_type => doc set on logstash output.

i also have index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" set but when i do this i receive the follow error from logstash

[2018-03-08T16:45:03,996][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-6.2.1-2018.03.08", :_type=>"doc", :_routing=>nil}, #LogStash::Event:0x46a05c23], :response=>{"index"=>{"_index"=>"filebeat-6.2.1-2018.03.08", "_type"=>"doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"Failed to parse mapping [doc]: Mapping definition for [error] has unsupported parameters: [properties : {code={type=long}, message={norms=false, type=text}, type={ignore_above=1024, type=keyword}}]", "caused_by"=>{"type"=>"mapper_parsing_exception", "reason"=>"Mapping definition for [error] has unsupported parameters: [properties : {code={type=long}, message={norms=false, type=text}, type={ignore_above=1024, type=keyword}}]"}}}}}

Any hints? I fear i have missed something.

Bloke

If anyone is interested. I deleted the older template for filebeat-* and used the latest filbeat agent/s 6.x+ Solved my issue. Also is a prompter for not letting to much versioning go by with these products or more and more issues occur with upgrading.

Note: I did save the older template as i was going to reload it with a greater order number but it also would require changing to be compatible with the ES 6.0 changes.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.