Statistical Facets starts returning inconsistent values (very large numbers 1E18)

Elastic Stack Elasticsearch
1

Hi,

We are running an 8 nodes cluster of elastic v1.5.
We are experiencing a strange behavior of statistical facets queries on one particular field (type double) of an index (4 shards, 1 replica).
It's starting to return very large values (like 1E18) for total, max, mean, sum_of_squares, variance and std_deviation. Min and count values are correct. It happens even on small doc count, we get the same result with sum aggregation.

Did anyone already experience this issue ?
How to fix it ? (close/open index, reindex, cluster restart)

Thanx in advance,
Guillaume

2

I think you got hit by this bug: https://github.com/elastic/elasticsearch/issues/8688 which will be fixed in 2.0. What probably happened is that you field got dynamically mapped as a long on a shard and a double on another shard, then shard moved and you either ended up interpreting long bits as doubles or double bits as longs. The only work-around in 1.x would be to map fields explicitely prior to indexing. Unfortunately in your case there is not other fix than reindexing.

3

Thank you Adrien.
Do you think we can use the documents of the actual index and reindex them in another one ? Strangely the values seem correct.

4

You can use the _source documents to reindex, the issues are only within the index and/or doc values.