I have two visualizations - both against the same saved search (and the same timeframe).
One is a Data table , that on top of some other summaries (which in this case includes all docs) , includes a Unique count metric of a not_analyzed field (host.ip) - which reports 314 unique ips.
The second is a simple Unique count metric visualization of the same field - however this visualization , which is not contrained by any other limitations , only reports 294 unique ips.
How is this possible ?
This is a test environment - running Elasticsearch and kibana on a single machine . no distributed shards or anything. Just ~3mil docs in a single index.
I did a further - interresting and worrying test.
(Note: the doc count has increased slightly , but its inconsequential - as its a fixed test data set that I replay over and over - thus the unique count of hosts is not affected)
.1 The same data table visualization as above , still indicating 314 unique IPs.
.2 Next, I removed the blank "alarm.rule_description" field from the data table - and surprisingly this caused the unique ip count to drop... this time to 289 , which is now below the '294' reported by the unique count metric visualization.
.3 Surprised by the above findings , I removed the "alarm.is_alarm" field next - and Again, the value changed. This time to the actual 294.
How can these two single value fields have an impact on the unique count even though the event count stays the same?
(In the 3rd image the event count did go up - because my program cycled again. But I did redo the test and confirmed the event count stays the same no matter which fields I have visible.)
Unique counts are based on the cardinality aggregation, which is designed to work efficiently across very large amounts of data and delivers an approximate result, which may explain why your results vary. It is possible to tune the precision used, and you should be able to do so by providing a 'precision_threshold' parameter through the JSON Input when building the visualisation.
1 Like
Ah - thank you very much. I'll have a look into that.
