Aggregate distinct value based on logs in kibana

[Nitesh_Chaudhary]

Hi,
I have logs collected in my kibana. Format is like below:
{timestamp: 'July 20th 2020, 16:17:55.029', operation: 'success', 'element': '1'}
{timestamp: 'July 20th 2020, 16:18:55.029', operation: 'success', 'element': '2'}
{timestamp: 'July 20th 2020, 16:19:55.029', operation: 'success', 'element': '3'}
{timestamp: 'July 20th 2020, 16:20:55.029', operation: 'success', 'element': '1'}
{timestamp: 'July 20th 2020, 16:20:57.029', operation: 'failure', 'element': '1'}
{timestamp: 'July 20th 2020, 16:20:58.029', operation: 'failure', 'element': '2'}
{timestamp: 'July 20th 2020, 16:21:58.029', operation: 'failure', 'element': '2'}
{timestamp: 'July 20th 2020, 16:22:58.029', operation: 'failure', 'element': '2'}

I want to count the number of unique element by operation type like,
operation: success , totalcount: 4, unique element count: '3'
operation: failure, totalcount: 4, unique element count: '2'

I tried using visualisation(metric count) but there you can collect distinct count on based of one parameter but not the nested one.(As in i can count total success or failure operation type but not the nested one)
Is there a way to make this data from log??

[tylersmalley]

I don't believe you're going to be able to show the unique elements without actually aggregating on them.

Here is the test data I have for anyone wanting to also try:

DELETE /discuss-241883

PUT /discuss-241883
{
    "settings" : {
        "index" : {
            "number_of_shards" : 1, 
            "number_of_replicas" : 0 
        }
    }
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:17:55.029",
    "operation" : "success",
    "element" : "1"
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:18:55.029",
    "operation" : "success",
    "element" : "2"
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:19:55.029",
    "operation" : "success",
    "element" : "3"
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:20:55.029",
    "operation" : "success",
    "element" : "1"
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:20:57.029",
    "operation" : "failure",
    "element" : "1"
}



POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:20:58.029",
    "operation" : "failure",
    "element" : "2"
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:21:58.029",
    "operation" : "failure",
    "element" : "2"
}

POST /discuss-241883/_doc
{
    "@timestamp" : "July 20th 2020, 16:22:58.029",
    "operation" : "failure",
    "element" : "2"
}

And a visualization for the error rate:

You could, however, split the table into success and failure and show the top X failed elements:

[Nitesh_Chaudhary]

thanks tylersmalley that works for me.