I also have the issue and following are the details of the issue.
We have upgraded filebeat and logstash to 6.3 as part of patching.
#logstash -V
logstash 6.3.0
#filebeat version
filebeat version 6.3.0 (amd64), libbeat 6.3.0
we noticed that the format of host through logstash has changed as follows:
e.g. "host":"xxx.ood.ops" to "host": { "name": "xxx.ood.ops" }
After doing the upgrade we are getting the following errors
In logstash:
Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"3320728454", :_index=>"logstash-2018.07.18", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x4e8bfcd8>], :response=>{"index"=>{"_index"=>"logstash-2018.07.18", "_type"=>"doc", "_id"=>"3320728454", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"object mapping for [host] tried to parse field [host] as object, but found a concrete value"}}}}
In elasticsearch:
[2018-07-18T17:01:04,903][DEBUG][o.e.a.b.TransportShardBulkAction] [logstash-2018.07.18][1] failed to execute bulk item (index) BulkShardRequest [[logstash-2018.07.18][1]] containing [11] requests
org.elasticsearch.index.mapper.MapperParsingException: object mapping for [host] tried to parse field [host] as object, but found a concrete value
The json format as seen in kibana:
{
"_index": "logstash-2018.07.18",
"_type": "doc",
"_id": "2114191840",
"_version": 102181,
"_score": null,
"_source": {
"prospector": {
"type": "log"
},
"host": {
"name": "xxx.ood.ops"
},
"source": "/var/log/sample.out",
"beat": {
"name": "xxx.ood.ops",
"hostname": "xxx.ood.ops",
"version": "6.3.0"
},
"input": {
"type": "log"
}
I tried the solution of renaming and following is the error:
[2018-07-18T16:01:10,186][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"4218739081", :_index=>"logstash-2018.07.18", :_type=>"doc", :_routing=>nil}, #LogStash::Event:0x14e8812e], :response=>{"index"=>{"_index"=>"logstash-2018.07.18", "_type"=>"doc", "_id"=>"4218739081", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [host.name]", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:57"}}}}}
Can you please help ?