Kibana nested JSON search

Elastic Stack Kibana
1

Currently we pipe application logs to ElasticSearch via Fleuntd. When searching it's been impossible to do so by JSON Keys within a log field. Below in our 'log' field we have a correctly formatted JSON string. However it seems impossible to search e.g. event:Request Received will not actually search the event field within the log field.

What would the correct syntax be here to do such sub searching? We also want to build queries around the meta data field that has its own subset of fields.

sampleEvent
sampleEvent.png1077×456 41 KB

Edit - I looked at the 'JSON' version of the data tab and saw that interestingly the data was being escaped out

"log": "{"meta": {"scheme": "http", "request_id": "d50e46f6-e9e3-486f-aba2-bc507904e9bd", "headers": {"User-Agent": "ELB-HealthChecker/2.0", "Connection": "close", "Host": "172"}, "method": "GET", "path": "/health?", "remote_addr": "172"}, "event": "Request Received", "name": "staff", "env": "prod", "time": "2017-08-18T15:02:53.132817Z"}\n",

Would this be an issue with FluentD or ElasticSearch itself?

1 Like
2

It looks like FluentD is sending the log line to Elasticsearch as a text string instead of a JSON. Does FluentD have a way or a plugin that will allow the log to be parsed and formatted as JSON?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.