I'm Getting wrong counts in version 8.5.0
Time range is selected absolute values.
The count of records are 102,118 hits
When I create it's widget metrics of count it's same as expected: 102118
When I create unique count widget i got 102274 .
How It can be true where as total documents are 102118 and it's unique count can not be greater than it's count of documents. Time range is selected same (fixed).Is anyone also getting same issue of calculation?
one possible answer to that could be that the unique count operation is applied to a field with array values in it.
Can you check that there are no array values within the fields.transaction_id (use Discover for that)?
If the field contains array values then there are more values than documents, hence the divergence of the two results.
It does not contain array. it's type is long. It works correctly if I narrow down it's time range where as if I increase time range I got wrong calculation not even this calculation but more widgets same like this in different indices.
Even It's latest transform index where I use fields.transaction_id as a unique key.
Another factor to keep in mind is that the ES Cardinality aggregation used under the hood for the unique count in Lens provides "an approximate count of distinct values" as documented: Cardinality aggregation | Elasticsearch Guide [8.9] | Elastic
More explanation:
AS you can see 30 mints data give correct result. time range: Jul 31, 2023 @ 00:00:00.000 to Jul 31, 2023 @ 01:00:00.000
AND Jul 31, 2023 @ 00:00:00.000 To Jul 31, 2023 @ 00:30:00.000
AS you can see answer is correct on 30 mints interval.
now I will compile both time 30 mints time interval and create widget of 1 hour. time Jul 31, 2023 @ 00:00:00.000 to Jul 31, 2023 @ 01:00:00.000
answer is different 4 document extra.
Try to open the Inspect, then go to the Request tab and click on Open in Console.
In the request, can you add a "precision_threshold": 15000 to the cardinality agg?
"cardinality": {
"field": "fields.transaction_id.keyword",
"precision_threshold": 15000
}
Then hit the send request and check the result: is that correct? If so, it's an approximation issue of the cardinality agg.
Hi @Marco_Liberati
It's giving the correct unique count after adding precision_threshold.
So how do I resolve it in the lens widget?
Without precision_threshold result is incorrect.
Any solution?
Thanks
"precision_threshold" is not something currently supported in Lens.
There's already a feature request to add it to the editor you can follow here: [Lens] Unique count aggregation should have control for precision threshold and warning about estimates · Issue #69832 · elastic/kibana · GitHub
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.