Unique Count Metric and Cardinality filter not providing the same result

Elastic Stack Elasticsearch
1

Hi,

in the first step i have set up some metric visualizations with in this case unique count on one specific field. As a next step i want to implement watcher on this field but the filter query is not providing the same result on the same filter ( for my understating they are equal).

my watcher filter is like:

		"adqry": {
              "search": {
                "request": {
                  "indices": [
                    "50005_myindex"
                  ],
                  "body": {
                    "query": {
                      "query_string": {
                        "query": "procname:adinfo AND _exists_:userprincipalname_hash AND NOT testevent:true AND -domain:/my.domain.+/ AND @timestamp:>now-24h"
                      }
                    },
                    "aggs": {
                      "type_count": {
                        "cardinality": {
                          "field": "userprincipalname_hash"
                        }
                      }
                    }
                  }
                }
              }
            }

The visualization is set up as follows:

  1. saved a discover filtering: procname:adinfo AND _exists_:userprincipalname_hash AND NOT testevent:true AND -domain:/my.domain.+/ AND @timestamp:>now-24h
  2. configured a metric visualizations refering on the prior saved discover with "metric-> aggregation: unique count" and "field: principalusername_hash"

My problem is, that the visualization provides: 72 623 as a unique count
and the filter provides

ctx.payload.adqry.hits.total: 75741

as a unique count
and i dont know what is wrong.

many thanks in advance and a sunny weekend!

2

Hey,

can you just run the query in dev tools and see what is being returned? I'd assume the same value than the one from watcher (including some fluctuations, because every query is going to have a different now).

As watcher is only executing that search query, I assume there might be a slight difference with the search query.

Is it possible that you are using Kuery in kibana as your query language? That would not be supported in watcher/elasticsearch.

--Alex

3

Hey Alex,

thanks for the reply.

the problem forcing the deviation was caused by not referencing on the corect payload value.

i needed to get the value from the cardinality aggregation but my payload was fetching the total value of hits -> ctx.payload.adqry.hits.total instead of the result of the aggregation -> ctx.payload.adqry.aggregations.count.value

cheers, daniel

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.