# After upgrading from FB 5.6.5 to FB 6.5.4, events stopped indexing **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [March 18, 2019, 7:18pm UTC](https://discuss.elastic.co/t/after-upgrading-from-fb-5-6-5-to-fb-6-5-4-events-stopped-indexing/172854 "2019-03-18T19:18:49Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Frederico\_Ferreira](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frederico_ferreira/32/42268_2.png) [@Frederico\_Ferreira](https://discuss.elastic.co/u/Frederico_Ferreira)\ **Post date:** [March 18, 2019, 7:18pm UTC](https://discuss.elastic.co/t/after-upgrading-from-fb-5-6-5-to-fb-6-5-4-events-stopped-indexing/172854/1 "2019-03-18T19:18:49Z")
Hello there. Before someone comes posting that this is a duplicate, the threads that I found here does not solve my issue. Here it goes: My stask works like this: Filebeat -\> Logstash -\> Elasticsearch I have lots and lots of Filebeats versions, from 1.2.3 to 6.6.\* and, to solve this, I'm trying to normalise all versions under 6.5.4 (the same version that of my Logstashs). The thing is, I have some indexes (I believe, created from FB 1.2.3) that, when I updated to v6.5.4, stopped indexing. If I go back to 5.6.\*, it works. Updating the Filebeat breaks indexing. The mapping of the index I'm working on right now: [https://pastebin.com/PHJuG1h9](https://pastebin.com/PHJuG1h9) The logs that I get from Logstash are: ```auto [2019-03-18T18:29:25,331][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"59f8bbf18d64dcc41bd1bd8eb60c73b87491c682", :_index=>"callback-2019.03.18", :_type=>"callbac k", :_routing=>nil}, #], :response=>{"index"=>{"_index"=>"callback-2019.03.18", "_type"=>"callback", "_id"=>"59f8bbf18d64dcc41bd1bd8eb60c73b87491c682", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception ", "reason"=>"failed to parse [host]", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:563"}}}}} ``` So (from the mapping)... ```auto "host" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } } ``` My ES version is: ```auto { "name" : "FPTaOlj", "cluster_name" : "252450725677:logs", "cluster_uuid" : "QixOjWG0QuqjcaZhLfFPQg", "version" : { "number" : "5.5.2", "build_hash" : "363575f", "build_date" : "2018-07-31T10:54:15.297Z", "build_snapshot" : false, "lucene_version" : "6.6.0" }, "tagline" : "You Know, for Search" } ``` My Logstash configuration: ```auto input { beats { port => 10095 add_field => { "hostname" => "%{[beat][hostname]}" } add_field => { "logstash-server" => "ip-{{ ansible_default_ipv4.address | replace('.', '-')}}" } } } filter { if "json" in [tags] { if [message] =~ /^\s*$/ { drop { } } if [message] =~ /^raven@.* alert: failed to send exception to sentry.*/ { drop { } } json { source => "message" } if [timestamp] { date { match => ["timestamp", "ISO8601","YYYY-MM-dd HH:mm:ss.SSSSSS"] } } } } output { elasticsearch { hosts => ["{{ es_url }}:{{ es_port }}"] index => "%{type}-%{+YYYY.MM.dd}" document_id => "%{message_fingerprint}" manage_template => false } } ``` My Filebeat config: ```auto filebeat: prospectors: - paths: - "/var/log/messages" - "/var/log/secure" - "/var/log/eb-activity.log" fields_under_root: true fields: type: "callback" tags: ["syslog", "staging", "callback-qa-new"] alert_email: "router@luc.id" - paths: - "/var/log/nginx/access.log" fields_under_root: true fields: type: "callback" tags: ["nginx", "staging", "callback-qa-new"] alert_email: "router@luc.id" - paths: - "/var/log/nodejs/nodejs.log" fields_under_root: true fields: type: "callback" tags: ["json", "staging", "callback-qa-new"] alert_email: "abcde@poiu.com" output: logstash: hosts: - "XXX.YYY.ZZZ.AAA:PPPP" loadbalance: true worker: 1 ``` Could you help me with that? ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [March 22, 2019, 9:38am UTC](https://discuss.elastic.co/t/after-upgrading-from-fb-5-6-5-to-fb-6-5-4-events-stopped-indexing/172854/2 "2019-03-22T09:38:01Z")
Have you looked at this breaking change introduced in FB 6.3? [https://www.elastic.co/guide/en/beats/libbeat/current/breaking-changes-6.3.html#breaking-changes-mapping-conflict](https://www.elastic.co/guide/en/beats/libbeat/current/breaking-changes-6.3.html#breaking-changes-mapping-conflict) ---
**Author:** ![Frederico\_Ferreira](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frederico_ferreira/32/42268_2.png) [@Frederico\_Ferreira](https://discuss.elastic.co/u/Frederico_Ferreira)\ **Post date:** [March 25, 2019, 4:10pm UTC](https://discuss.elastic.co/t/after-upgrading-from-fb-5-6-5-to-fb-6-5-4-events-stopped-indexing/172854/3 "2019-03-25T16:10:08Z")
Thanks. That helps a lot! I'll try those tips as soon as I get my hands on this again. ---
**Author:** ![Frederico\_Ferreira](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frederico_ferreira/32/42268_2.png) [@Frederico\_Ferreira](https://discuss.elastic.co/u/Frederico_Ferreira)\ **Post date:** [March 27, 2019, 3:33pm UTC](https://discuss.elastic.co/t/after-upgrading-from-fb-5-6-5-to-fb-6-5-4-events-stopped-indexing/172854/4 "2019-03-27T15:33:16Z")
Thanks @kvch! It did work. What I did was to add this filter to Logstash: ```auto filter { mutate { remove_field => ["[host]" ] } } ``` I did try to add `drop_field` to Filebeat but it didn't work. Only the filter worked. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 24, 2019, 3:33pm UTC](https://discuss.elastic.co/t/after-upgrading-from-fb-5-6-5-to-fb-6-5-4-events-stopped-indexing/172854/5 "2019-04-24T15:33:18Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.