Filebeat 6.2.1 and logstash 6.2.1 and doc type


#1

Hi,

I recently updated ELK from 5.4 to 6.2.1

My upgrade went reasonably well and i modified some custom templates on the elasticsearch side in relation to SRX data and geoip. That all works.

My question is really in regard to filebeat 6.2.1 and logstash 6.2.1
I currently have 2 x filebeat templates in ES. Those are the "version" : "5.4.2" and "filebeat-6.2.1" : {
"order" : 1,
"index_patterns" : [
"filebeat-6.2.1-*"
],

In fear of boring you i wont post them here, the templates, which are as best i can see, the templates provided by default and installation of filebeats.

Those templates have a index-patterns and an order. Does 1 of these "the older" require deletion?

I read about the 6.0.0 and type being doc for logstash to es and compatibility with beats.

I have the document_type => doc set on logstash output.

i also have index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" set but when i do this i receive the follow error from logstash

[2018-03-08T16:45:03,996][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-6.2.1-2018.03.08", :_type=>"doc", :_routing=>nil}, #LogStash::Event:0x46a05c23], :response=>{"index"=>{"_index"=>"filebeat-6.2.1-2018.03.08", "_type"=>"doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"Failed to parse mapping [doc]: Mapping definition for [error] has unsupported parameters: [properties : {code={type=long}, message={norms=false, type=text}, type={ignore_above=1024, type=keyword}}]", "caused_by"=>{"type"=>"mapper_parsing_exception", "reason"=>"Mapping definition for [error] has unsupported parameters: [properties : {code={type=long}, message={norms=false, type=text}, type={ignore_above=1024, type=keyword}}]"}}}}}

Any hints? I fear i have missed something.

Bloke


#2

If anyone is interested. I deleted the older template for filebeat-* and used the latest filbeat agent/s 6.x+ Solved my issue. Also is a prompter for not letting to much versioning go by with these products or more and more issues occur with upgrading.

Note: I did save the older template as i was going to reload it with a greater order number but it also would require changing to be compatible with the ES 6.0 changes.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.