Cardinality and Field Data

(Eric Brandes) #1

We're on the 1.4 series of ES and using cardinality aggregations for a few things. Recently we switched to using a hashed field for better performance. As part of that, we noticed that the hashed field is being stored in field data. I was under the impression the hyper log log implementation was specifically brought in to avoid this? The docs state that the cardinality agg uses a fixed memory set, but we're seeing field data rise with the cardinality of the field. Am I misunderstanding how that agg should work? We do no other aggs on that field, and we have confirmed that it's the cardinality agg specifically that causes it.

(Maxim Valyanskiy) #2

Cardinality aggregation does not need extra memory to count distinct values. But field data is still needed to read values.

(system) #3