Kibana nested JSON search


(Ali Khan) #1

Currently we pipe application logs to ElasticSearch via Fleuntd. When searching it's been impossible to do so by JSON Keys within a log field. Below in our 'log' field we have a correctly formatted JSON string. However it seems impossible to search e.g. event:Request Received will not actually search the event field within the log field.

What would the correct syntax be here to do such sub searching? We also want to build queries around the meta data field that has its own subset of fields.

Edit - I looked at the 'JSON' version of the data tab and saw that interestingly the data was being escaped out

"log": "{"meta": {"scheme": "http", "request_id": "d50e46f6-e9e3-486f-aba2-bc507904e9bd", "headers": {"User-Agent": "ELB-HealthChecker/2.0", "Connection": "close", "Host": "172"}, "method": "GET", "path": "/health?", "remote_addr": "172"}, "event": "Request Received", "name": "staff", "env": "prod", "time": "2017-08-18T15:02:53.132817Z"}\n",

Would this be an issue with FluentD or ElasticSearch itself?


(Tim Sullivan) #2

It looks like FluentD is sending the log line to Elasticsearch as a text string instead of a JSON. Does FluentD have a way or a plugin that will allow the log to be parsed and formatted as JSON?


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.