How to improve the accuracy of Unique Count metric in Kibana

Elastic Stack Kibana
1

Hi,

I am seeing some differences in the counts.

Background:

  • Number of documents: 1,384,896
  • Unique counts of fields A, B and C are reported as : 3641, 3791, 3799
  • With {"precision_threshold":40000} Unique counts of fields A, B and C : 3637, 3791, 3789

The actual expected values are : 3638, 3791, 3791.

I tried to put a filter so as to reduce the number of documents from which the unique counts is taken. But it made no difference.

Will throwing more hardware at it help?

2

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-cardinality-aggregation.html

Guess you have read that.

Depending on what you are doing exactly you can use Vega to get exact counts and create a metric visualization.

I swapped all of our metrics over to Vega due to this reason. But if your count is extremely high then it might not work.

3

That is kind of sad. As we are trying to use it more we are finding more limitations. We cannot use it now for one of our most important metrics, e.g. First pass yield.

4

Tried with number of shards equals one and number of replicas set to zero. Still no luck on improving the error margin.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.