Logstash upgrade issue (Upgrade from 7.17.5 to 8.13.24

Elastic Stack Logstash
1

Hello Team,
Last week I upgraded Elasticsearch from 7.XX to 8.13.4. The Logstash version is 7.17.5. Everything is working as expected.

Now I tried to upgrade Logstash from 7.17.5 to 8.13.4 and it failed with the below error and logs stopped pushing. The host field is required and is type in Kibana. I have already tried Failed to parse field [host] of type [text] .. can't get text on a START_OBJECT at 1:974 - #2 by Badger. But it's just replacing the host as [host][name] instead of actual host . Any possible fixes or docs for the troubleshooting purpose?

[2024-06-11T07:26:52,902][WARN ][logstash.outputs.elasticsearch][devicelogs-amqp-elasticsearch-ingest][d57e8c46da4ff562ec20d6908f749056080bc80375b741104f12fafd3e7a18cd] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"xxxxxxxx", :routing=>nil}. response=>{"index"=>{"status"=>400, "error"=>{"type"=>"document_parsing_exception", "reason"=>"[1:2201] failed to parse field [host] of type [text] in document with id 'NOMxBpABkFDuo1q1TW3V'. Preview of field's value: '{hostname=20264237152408710}'", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:2170"}}}}}
2

Did you check the breaking changes before upgrading? This is a breaking change, it is related to the fact that pipeline.ecs_compatibility is now enabled per default in Logstash 8.X.

This means that it wil use ecs fields in the events and the host field is an object, but in your index it is mapped wrong.

You didn't share what are your inputs, some inputs will add the host field as an object now.

To solve this add pipeline.ecs_compatibility: disabled to your logstash.yml and restart it, this will bring back the old behavior regarding ecs fields.

But you will need to fix your mappings in the future.

3

Thank you for the help @leandrojmp