Tried to parse field as object, but found a concrete value

[Sirius (Jorge Alfredo Sánchez Lamas)]

Hi,

I'm running filebeat 7.6.2 with logstash 7.5.1 having missing lines of logs in ES due this warning message ...

[2020-04-21T21:25:21,105][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"tableau_json_log_test-2020.04", :routing=>nil, :_type=>"_doc"}, #LogStash::Event:0x2c15edaa], :response=>{"index"=>{"_index"=>"tableau_json_log_test-2020.04", "_type"=>"_doc", "_id"=>"W_OhnnEBf8OpJ0AbkmlE", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"object mapping for [v] tried to parse field [v] as object, but found a concrete value"}}}}

Data sample below it works with line #1 for [v] field structure but no for the #2,, any idea/advise how can I handle when [v] as an object ?

#1 :
{"ts":"2020-03-17T00:00:48.950","pid":18640,"tid":"6540","sev":"info","req":"-","sess":"-","site":"-","user":"-","k":"rotate-log","v":{"new-path":"D:\Tableau\Tableau Server\data\tabsvc\logs\vizqlserver\nativeapi_vizqlserver_9-2_2020_03_17_00_00_00.txt","old-path":"D:\Tableau\Tableau Server\data\tabsvc\logs\vizqlserver\nativeapi_vizqlserver_9-2_2020_03_16_00_00_00.txt"}}

#2 :
{"ts":"2020-03-17T00:00:48.949","pid":18640,"tid":"5cb0","sev":"info","req":"-","sess":"-","site":"-","user":"-","k":"msg","v":"ModelCacheInvalidator notifying dirty model for InvalidatorId=4270"}

I'm using these properties in my input filebeat to read json log files, the rest configuration is the default one.

json.keys_under_root: true
json.add_error_key: true
json.message_key: log

logstash configuration is:

input {
beats {
port => 5040
ssl => true
ssl_key => '..../logstash-test.pkcs8.key'
ssl_certificate => '..../logstash-test.crt'
}
}
output {
if [fields][log_type] == "tableau_json_log_test" {
elasticsearch {
hosts => ["https://test1.com:9200","https://test2.com:9200","https://test3.com:9200"]
index => "tableau_json_log_test-%{+YYYY.MM}"
ssl => true
ssl_certificate_verification => true
cacert => '.../ca.crt'
user => 'usr'
password => 'passwd'
}
}
}

[Luca_Belluccini]

This is a mapping conflict.
If you do not use any index template with a index mapping, you might get conflicts.
Still, in your specific case it would not be enough.

In some documents, it seems v is a scalar value (e.g. a string).
In other documents, it contains an object.

A way to solve this might be to rename the v field to be different depending on k.

This is only applicable if the number of different values of k is limited (e.g. ~100), otherwise you might risk field explosion.

Using Logstash:

mutate {
  rename => ["v", "%{[k]}" ]
}

Using this mutate, the log #1 becomes:

{"ts":"2020-03-17T00:00:48.950","pid":18640,"tid":"6540","sev":"info","req":"-","sess":"-","site":"-","user":"-","k":"rotate-log","rotate-log":{"new-path":"D:\Tableau\Tableau Server\data\tabsvc\logs\vizqlserver\nativeapi_vizqlserver_9-2_2020_03_17_00_00_00.txt","old-path":"D:\Tableau\Tableau Server\data\tabsvc\logs\vizqlserver\nativeapi_vizqlserver_9-2_2020_03_16_00_00_00.txt"}}

Using this mutate, the log #2 becomes:

{"ts":"2020-03-17T00:00:48.949","pid":18640,"tid":"5cb0","sev":"info","req":"-","sess":"-","site":"-","user":"-","k":"msg","msg":"ModelCacheInvalidator notifying dirty model for InvalidatorId=4270"}

[Sirius (Jorge Alfredo Sánchez Lamas)]

Thanks so much Luca, it resolved the issue !!
And yes I'm running now over field limited warnings,, I'll be defining a template for mapping those field useful.