HELP: Grouping and combine fields

Elastic Stack Kibana
1

Hi All,

I am bit new to kibana.
Till now using fields i was able to bulid my visualization and dashboard. But now my requirement is to group fields and show it in PIE chart which i feel it needs some advance query knowledge, which i have never done.

My requirement is like below: I have multiple entries like below
ABC-11-2019
ABC-12-2019
ABC-01-2020
XYZ-11-2019
XYZ-12-2019
XYZ-01-2020
QWE-11-2019
QWE-12-2019
QWE-01-2020

It has entries which ends with month and year.

In my pie chart I need to show only 3 slices ABC, XYZ and QWE, however when i try to do it shows 9 slices.

Can someone point me to the right direction, how I can write query to build right visualization as per the requirement.

Appreciate all your help.

2

@bharatchhajed The reason why it is showing 9 is because when you do a terms aggregation (for piechart) , each of these entries are unique terms (ABC-11-2019 for e.g. is not the same as ABC-01-2019). So the total unique terms turns out to be 9 and not 3.

If you are using Logstash to ingest your logs, you could parse that field with something like below (if ABC or XYZ is always going to be in the beginning)

ruby {
    code => "event.set('prefix', event.get('value').split('-').first)"
  }

and then use the newly created 'prefix' field in your logs for doing a terms aggregation on it. If you already have ingested logs in your Elasticsearch, you will have to reindex logs while running it through that parser code.

3

Thanks Rahul for your reply. We are not using Logstash, we are injecting logs to ES directly from our application.
Is there a way to handle from Kibana side by writing query or something.

I tried doing {"script":"doc['_index'].value.substring(0,2)"} its not working all the time as someof the fields have more then 3 charecters. Is there a way where I can do a split or some string manipulation where i can split with -.
I also tried {"script":"doc['_index'].value.split('-').first"} --> This has not worked

Thanks in advance

4

@bharatchhajed You can try to create a Kibana scripted field with the script below

if (doc['value.keyword'].size() == 0) {
    return 'NA'; 
}else {
    String str = doc['value.keyword'].value;
    String id =str.substring(0,str.indexOf("-")); 
    return id; 
}

You can change NA to whatever you want. This can serve as a bucket for those docs that don't have that field if you want analyze them in your pie chart.

Please change the field name from value to whatever your field name is. When creating the scripted field in Kibana, choose the type of field to be String

1 Like
5

@bharatchhajed This approach works when I try in my ES/Kibana

Screen Shot 2020-03-07 at 3.32.14 PM
Screen Shot 2020-03-07 at 3.32.14 PM3236×1488 299 KB

6

Thanks a lot Rahul.. It works for me as well..
Cheers!

7

Hi Rahul,
Can you please point out which language does it support? I tried couple of ways but was unable to do it. Does it support split and arrays as well?
Thanks in advance :slight_smile:

8

Painless is the language you are looking for and below are some useful guides.

https://www.elastic.co/guide/en/kibana/current/scripted-fields.html

1 Like
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.