I have a log file in which each line is a syntactically-correct JSON object.
The problem is that there is a particular field that can either be a simple string or another JSON object (see the field "v" in the example below. In the actual file each json object is a single line: I've just cleaned up the formatting here for easier reading).
This appears to anger Elasticsearch, which doesn't like the inconsistency of the field type (note, this message doesn't match the example JSON, but if I could find the message it would relate to the 2nd JSON object):
[2017-12-23T20:58:29,602][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"sit-vizql-2017-12-12", :_type=>"log", :_routing=>nil}, 2017-12-12T15:14:39.048Z xxxxxx %{message}], :response=>{"index"=>{"_index"=>"sit-vizql-2017-12-12", "_type"=>"log", "_id"=>"AWCFK3_hbzJ_BvWdpTDK", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"object mapping for [v] tried to parse field [v] as object, but found a concrete value"}}}}
I would be grateful for any suggestions. Do I need to manually mangle the "v" field?
{
"ts": "2017-12-19T15:22:07.026",
"pid": 1976,
"tid": "d98",
"sev": "info",
"req": "-",
"sess": "-",
"site": "-",
"user": "-",
"k": "PathAccessChecker",
"v": {
"this": "0x000000001b93c1b0",
"matching-rules": [
{
"index": "1",
"path": "*",
"allowed": "1",
"type": "allowed-config"
}
],
"allowall-from-config": "1"
}
}
{
"ts": "2017-12-19T15:22:07.026",
"pid": 1976,
"tid": "d98",
"sev": "info",
"req": "WjkungoDcUEAAA4ENzwAAAO1",
"sess": "-",
"site": "-",
"user": "-",
"k": "msg",
"v": "ACTION: New Workbook"
}