How to use JSON Input to make filter on part of log

Elastic Stack Kibana
1

I have collected logs which are already imported into ELK.
Each log contains several fields, for ex date is different field.
Also there is complicated field "Details" which consist a lot of data: method: value, source: value, instrument: value and so on.
When I do histogram a choose number of logs on y-axis and date on x-axis.
So I get a lot of columns: from 0 to 100 of events (logs) in each column.

Question. Can I somehow do split series using sub aggregation adding as a filter instrument? I want to see each column with instruments also in addition to number of events each time period.

Example:
Log1
Date ddmmyy, hh:ss
…
Details: source: A, instrument: Car
…

Log2
Date ddmmyy, hh:ss
…
Details: source: A, instrument: Bike
…

How to make sub aggregation with instrument value?
Is it possible with JSON input: split series, filters, advanced, json input?

Thanks.

2

No ideas? Should I go to use script to create new fields that I need? It will update thousands and thousands of logs per day. :frowning:

3

Sounds like ideally you'd use a nested fields type for it though unfortunately we're just building out support for nested fields now so it's not ready for prime time in visualizations.

You can follow along on this issue:

In the mean time, ideally you'd breakup the field before doing anything with it. If you're using logstash, it has a split filters plugin which can be used to do exactly this: https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html#plugins-filters-split

4

Thanks, got it.
So as for now i'm working on rebuilding of my logs.
GL with nested fields.