Custom log file Filtering

[RickJC]

Hi All,

I'm a newbie so sorry if my question is silly. I've just started out with a ELK stack which works great (6.1) and using filebeat to ship logs. My question is, I want to filter out the "message" part of the log sent but i'm getting a little confused about how to do it.

I've been reading through about grok patterns which is fine but the custom log file in question for the "message" field is several lines long and i want to filter this so i can search via it (or perhaps grep the small important info from the message?). I've tried some of the debugger sites but no luck on what i need to do.

If someone can point me in the right direction that would be great.

[warkolm (Mark Walkom)]

Can you post an example?

[RickJC]

I've edited the log a little but as you can see there are duplicate 'fields' which is what i want to be able to search by

[2018-02-06 08:38:54.484699] 32106 DEBUG (H:testing01:12345) Workload: { "location_report": { "device_locations": [ { "id": 123456789, "device_id": 12345, "wearer_id": 123, "eventtime": "2018-02-06 08:23:58", "lat": 12.123456, "lon": -1.123456, "lac": 12345, "db": "2018-02-06 08:38:54" }, { "id": 123456789, "device_id": 12345, "wearer_id": 123, "eventtime": "2018-02-06 08:24:50", "lat": 12.123456, "lon": -1.123456, "lac": 12345, "db": "2018-02-06 08:38:54" } ] } }

[RickJC]

Just wondering if anyone had any ideas?