# Aggregate distinct value based on logs in kibana

**URL:** <https://discuss.elastic.co/t/aggregate-distinct-value-based-on-logs-in-kibana/241883>\
**Category:** Kibana\
**Created:** [July 20, 2020, 10:57am UTC](https://discuss.elastic.co/t/aggregate-distinct-value-based-on-logs-in-kibana/241883 "2020-07-20T10:57:40Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nitesh\_Chaudhary](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nitesh_chaudhary/32/72435_2.png) [@Nitesh\_Chaudhary](https://discuss.elastic.co/u/Nitesh_Chaudhary)\
**Post date:** [July 20, 2020, 10:57am UTC](https://discuss.elastic.co/t/aggregate-distinct-value-based-on-logs-in-kibana/241883/1 "2020-07-20T10:57:41Z")

</div>

Hi,  
I have logs collected in my kibana. Format is like below:  
{timestamp: 'July 20th 2020, 16:17:55.029', operation: 'success', 'element': '1'}  
{timestamp: 'July 20th 2020, 16:18:55.029', operation: 'success', 'element': '2'}  
{timestamp: 'July 20th 2020, 16:19:55.029', operation: 'success', 'element': '3'}  
{timestamp: 'July 20th 2020, 16:20:55.029', operation: 'success', 'element': '1'}  
{timestamp: 'July 20th 2020, 16:20:57.029', operation: 'failure', 'element': '1'}  
{timestamp: 'July 20th 2020, 16:20:58.029', operation: 'failure', 'element': '2'}  
{timestamp: 'July 20th 2020, 16:21:58.029', operation: 'failure', 'element': '2'}  
{timestamp: 'July 20th 2020, 16:22:58.029', operation: 'failure', 'element': '2'}

I want to count the number of unique element by operation type like,  
operation: success , totalcount: 4, unique element count: '3'  
operation: failure, totalcount: 4, unique element count: '2'

I tried using visualisation(metric count) but there you can collect distinct count on based of one parameter but not the nested one.(As in i can count total success or failure operation type but not the nested one)  
Is there a way to make this data from log??

---

<div class="post-metadata">

**Author:** ![tylersmalley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tylersmalley/32/8833_2.png) [@tylersmalley](https://discuss.elastic.co/u/tylersmalley)\
**Post date:** [July 20, 2020, 10:32pm UTC](https://discuss.elastic.co/t/aggregate-distinct-value-based-on-logs-in-kibana/241883/2 "2020-07-20T22:32:33Z")

</div>

I don't believe you're going to be able to show the unique elements without actually aggregating on them.

Here is the test data I have for anyone wanting to also try:

```auto
DELETE /discuss-241883

PUT /discuss-241883
{
    "settings" : {
        "index" : {
            "number_of_shards" : 1, 
            "number_of_replicas" : 0 
        }
    }
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:17:55.029",
    "operation" : "success",
    "element" : "1"
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:18:55.029",
    "operation" : "success",
    "element" : "2"
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:19:55.029",
    "operation" : "success",
    "element" : "3"
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:20:55.029",
    "operation" : "success",
    "element" : "1"
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:20:57.029",
    "operation" : "failure",
    "element" : "1"
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:20:58.029",
    "operation" : "failure",
    "element" : "2"
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:21:58.029",
    "operation" : "failure",
    "element" : "2"
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:22:58.029",
    "operation" : "failure",
    "element" : "2"
}

```

And a visualization for the error rate:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/d/5d78251e2f0e6b7122edcb968ef1ee68f26d0f5f.png)

You could, however, split the table into success and failure and show the top X failed elements:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/3/e3eb345a30dcd699ec87dc15c59f3d1b4a33443c.png)

---

<div class="post-metadata">

**Author:** ![Nitesh\_Chaudhary](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nitesh_chaudhary/32/72435_2.png) [@Nitesh\_Chaudhary](https://discuss.elastic.co/u/Nitesh_Chaudhary)\
**Post date:** [July 21, 2020, 6:55am UTC](https://discuss.elastic.co/t/aggregate-distinct-value-based-on-logs-in-kibana/241883/3 "2020-07-21T06:55:17Z")

</div>

thanks tylersmalley that works for me.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 18, 2020, 6:55am UTC](https://discuss.elastic.co/t/aggregate-distinct-value-based-on-logs-in-kibana/241883/4 "2020-08-18T06:55:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.