# Create and show aggregated data

**URL:** <https://discuss.elastic.co/t/create-and-show-aggregated-data/109015>\
**Category:** Kibana\
**Created:** [November 24, 2017, 10:52am UTC](https://discuss.elastic.co/t/create-and-show-aggregated-data/109015 "2017-11-24T10:52:48Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![anujjain0801](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@anujjain0801](https://discuss.elastic.co/u/anujjain0801)\
**Post date:** [November 24, 2017, 10:52am UTC](https://discuss.elastic.co/t/create-and-show-aggregated-data/109015/1 "2017-11-24T10:52:48Z")

</div>

Hi Team,  
I have ELK setup in my VMs and I am able to see application logs in kibana. Now I want to aggregate data based on correlation Ids and then when user clicks on any such correlation ID it should show details of the logs for that correlation id(Basically like a hyperlinks where the correaltion ids will be showed and on click of the link details of those corrids should be seen). Can someone please help me if this is possible in Kibana and if yes how can I achieve it?

Any help is greatly appreciated!!

Thanks and Regards,  
Anuj Jain

---

<div class="post-metadata">

**Author:** ![ppisljar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppisljar/32/11588_2.png) [@ppisljar](https://discuss.elastic.co/u/ppisljar)\
**Post date:** [November 24, 2017, 11:03am UTC](https://discuss.elastic.co/t/create-and-show-aggregated-data/109015/2 "2017-11-24T11:03:31Z")

</div>

I would suggest to put your visualization, with aggregated data on your dashboard and next to it put a saved search which will be showing all your records formatted in a way you like.

clicking on id in visualization will then filter the saved search to only that id.

you can save your search in the discovery app and then use it on your dashboard just as you would visualization.

---

<div class="post-metadata">

**Author:** ![anujjain0801](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@anujjain0801](https://discuss.elastic.co/u/anujjain0801)\
**Post date:** [November 24, 2017, 11:08am UTC](https://discuss.elastic.co/t/create-and-show-aggregated-data/109015/3 "2017-11-24T11:08:58Z")

</div>

Hi Peter,  
Thank you so much for responding. I am very new to Kibana. Can you please give me some example how can I do it? I have app logs in message attribute which contains all the info like correlation id and other details. How Can I aggregate all the logs from the message attribute in elastic and display in Kibana.

Really appreciate your reply .

Thanks and Regards,  
Anuj Jain

---

<div class="post-metadata">

**Author:** ![ppisljar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppisljar/32/11588_2.png) [@ppisljar](https://discuss.elastic.co/u/ppisljar)\
**Post date:** [November 24, 2017, 11:23am UTC](https://discuss.elastic.co/t/create-and-show-aggregated-data/109015/4 "2017-11-24T11:23:27Z")

</div>

I dont fully understand your question.  
Do you mean you have just one field with all the information inside ? If that is the case you should first index your data in a way that correlation id is a separate field, else you will not be able to aggregate on it at all.

Take a look at logstash documentation on how you can parse your logs: [https://www.elastic.co/guide/en/logstash/current/index.html](https://www.elastic.co/guide/en/logstash/current/index.html)

and if you have issues with that try asking in the logstash forum:

> **[Logstash](https://discuss.elastic.co/c/logstash)**
>
> Everything related to your favorite centralized logging platform, including plugins and recipes.

if your data is already indexed correctly, please try to explain better what exactly would you like to achieve

---

<div class="post-metadata">

**Author:** ![anujjain0801](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@anujjain0801](https://discuss.elastic.co/u/anujjain0801)\
**Post date:** [November 24, 2017, 11:35am UTC](https://discuss.elastic.co/t/create-and-show-aggregated-data/109015/5 "2017-11-24T11:35:44Z")

</div>

Hi Peter,  
Thank you so much for the response. Please find a json attribute as below: -

`"message": "[2017-11-23 02:13:17.513] - debug: [corrId: xxxxxxxx][Response] {\"id\":\"EC-xxxxxx\",\"state\":\"APPROVED\",\"cart_id\":\"EC-XXXXXXXXXXX\",\"shipping_address\":{\"recipient_name\":\"XXXXXX\",\"id\":\"XXXXXXXXX\",\"line1\":\"XXXXXXx, XXXX\",\"line2\":\"XXXXX\",\"city\":\"xxxxx\",\"state\":\"xxx\",\"postal_code\":\"xxxxx-100\",\"country_code\":\"xx\",\"normalization_status\":\"UNKNOWN\",\"type\":\"GIFT\",\"default_address\":true,\"preferred_address\":false,\"primary_address\":false,\"disable_for_transaction\":false},\"payer\":{\"payment_method\":\"paypal\",\"status\":\"UNVERIFIED\",\"payer_info\":{\"email\":\"buyer@plus.paypal.com\",\"first_name\":\"xxxx\",\"last_name\":\"xxxx\",\"payer_id\":\"xxxxxxx\",\"shipping_address\":{\"recipient_name\":\"xxxxx\",\"id\":\"xxxxxx\",\"line1\":\"xx. xx, xxx\",\"line2\":\"apt xxx\",\"city\":\"xxx\",\"state\":\"xx\",\"postal_code\":\"xxxx-xx\",\"country_code\":\"xx\",\"normalization_status\":\"UNKNOWN\",\"type\":\"GIFT\",\"default_address\":true,\"preferred_address\":false,\"primary_address\":false,\"disable_for_transaction\":false},\"phone\":\"+xx xxxxxx\",\"phone_type\":\"HOME\",\"tax_id_type\":\"xxxxx\",\"tax_id\":\"xxxxxx\",\"country_code\":\"xx\",\"user_type\":\"GUEST\"},\"funding_option\":{\"id\":\"xxxxx\",\"rank\":0,\"funding_sources\":[{\"funding_mode\":\"INSTANT_TRANSFER\",\"funding_instrument_type\":\"PAYMENT_CARD\",\"amount\":{\"value\":\"xx\",\"currency\":\"xx\"},\"soft_descriptor\":\"AUTOMETALRE\",\"payment_card\":{\"id\":\"xxxxxx\",\"type\":\"xxxx\",\"number\":\"xxx\",\"card_product_class\":\"UNKNOWN\"}}]},\"funding_options\":[{\"id\":\"xxxx\",\"rank\":0,\"funding_sources\":[{\"funding_mode\":\"INSTANT_TRANSFER\",\"funding_instrument_type\":\"xxxx\",\"amount\":{\"value\":\"xxx\",\"currency\":\"xxx\"},\"soft_descriptor\":\"xxx\",\"payment_card\":{\"id\":\"xxxx\",\"type\":\"AMEX\",\"number\":\"xxx\",\"card_product_class\":\"UNKNOWN\"}}]}]},\"payment_approved\":xxx}"`

From the above such attribute I need to get the corrID value and aggregate data based on that corrId and create a hyper link where if any one clicks it will display data related to that corrId. I know we can do a string search that will display the data related to corrid. But what I need is I need to group data based on corrid and display all the available corrids in a particular time frame as hyper links and when any such hyperlink is clicked it will display messages related to that corrid.

Please let me know if you need more information.

Thanks and Regards,  
Anuj Jain

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 24, 2017, 11:42am UTC](https://discuss.elastic.co/t/create-and-show-aggregated-data/109015/6 "2017-11-24T11:42:39Z")

</div>

That is a string containing a JSON document. In order to analyse based on those fields they need to be indexed separately, so you need to parse this field before indexing the data. There are a number of options available:

- Use a [JSON filter](https://www.elastic.co/guide/en/logstash/6.0/plugins-filters-json.html) in Logstash
- Use a [JSON processor](https://www.elastic.co/guide/en/elasticsearch/reference/6.0/json-processor.html) within an ingest node pipeline
- If you are using Filebeat to ingest this data, it can also be configured [to decode the JSON data](https://www.elastic.co/guide/en/beats/filebeat/6.0/decode-json-fields.html) before indexing it into Elasticsearch.

---

<div class="post-metadata">

**Author:** ![anujjain0801](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@anujjain0801](https://discuss.elastic.co/u/anujjain0801)\
**Post date:** [November 24, 2017, 11:54am UTC](https://discuss.elastic.co/t/create-and-show-aggregated-data/109015/7 "2017-11-24T11:54:38Z")

</div>

Hi Christian,

Thank you so much for the response.  
I will try to explore the options you mentioned. After Indexing how can I aggregate and view data in Kibana?

Really appreciate your help on that.

Thanks and Regards,  
Anuj Jain

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 24, 2017, 12:18pm UTC](https://discuss.elastic.co/t/create-and-show-aggregated-data/109015/8 "2017-11-24T12:18:05Z")

</div>

Have a look at [the links provided in the getting started guide](https://www.elastic.co/guide/en/kibana/6.0/getting-started.html).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 22, 2017, 12:18pm UTC](https://discuss.elastic.co/t/create-and-show-aggregated-data/109015/9 "2017-12-22T12:18:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
