# Percentiles aggregation does not behave as expected **URL:** \ **Category:** Elasticsearch\ **Created:** [January 20, 2019, 7:52pm UTC](https://discuss.elastic.co/t/percentiles-aggregation-does-not-behave-as-expected/164962 "2019-01-20T19:52:16Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![iamredlus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iamredlus/32/15504_2.png) [@iamredlus](https://discuss.elastic.co/u/iamredlus)\ **Post date:** [January 20, 2019, 7:52pm UTC](https://discuss.elastic.co/t/percentiles-aggregation-does-not-behave-as-expected/164962/1 "2019-01-20T19:52:16Z")
Hey I've been experimenting with percentile aggregations on some dataset after receiving some odd results in elasticsearch 6.3.2. My dataset consists of 467 floating point numbers, which can be found [here](https://pastebin.com/vJxrcq5u). Just to rule out different implementations of the analyzed values, I've ingested this dataset (each value as a single document) into two indices - one which defines the field as a `float`, and one which defines the field as a `scaled_float` with `"scaling_factor": 1000`. The percentiles requested are `"percents": [50, 90, 95, 99, 99.99]`. In both cases, the results are very similar and are very different from the percentiles expected (vs. Python's NumPy): ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/7/573f634d178655352b9fa9f4fb307f63cf4df080.png) As documented, I know [percentiles are approximated](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-percentile-aggregation.html#search-aggregations-metrics-percentile-aggregation-approximation). However, it seems that the approximation behaves exactly opposite to what is written in the documentation: > - Accuracy is proportional to `q(1-q)` . This means that extreme percentiles (e.g. 99%) are more accurate than less extreme percentiles, such as the median > - For small sets of values, percentiles are highly accurate (and potentially 100% accurate if the data is small enough). My dataset is pretty small (only 467 samples), and the accuracy increases as we approach the median. How can this be explained? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [January 21, 2019, 7:49am UTC](https://discuss.elastic.co/t/percentiles-aggregation-does-not-behave-as-expected/164962/2 "2019-01-21T07:49:57Z")
How many shards does your index have? If you are using the default 5 primary shards, can you try with a single primary shard as well? ---
**Author:** ![iamredlus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iamredlus/32/15504_2.png) [@iamredlus](https://discuss.elastic.co/u/iamredlus)\ **Post date:** [January 21, 2019, 9:46am UTC](https://discuss.elastic.co/t/percentiles-aggregation-does-not-behave-as-expected/164962/3 "2019-01-21T09:46:04Z")
Hey @Christian_Dahlqvist, in both cases it was tested on a single shard. ---
**Author:** ![iamredlus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iamredlus/32/15504_2.png) [@iamredlus](https://discuss.elastic.co/u/iamredlus)\ **Post date:** [January 23, 2019, 10:46am UTC](https://discuss.elastic.co/t/percentiles-aggregation-does-not-behave-as-expected/164962/4 "2019-01-23T10:46:40Z")
Hey @Christian_Dahlqvist What else can we check on the issue? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [January 23, 2019, 11:23am UTC](https://discuss.elastic.co/t/percentiles-aggregation-does-not-behave-as-expected/164962/5 "2019-01-23T11:23:34Z")
Have you tried the [settings to balance memory usage and accuracy](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-percentile-aggregation.html#search-aggregations-metrics-percentile-aggregation-compression)? ---
**Author:** ![iamredlus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iamredlus/32/15504_2.png) [@iamredlus](https://discuss.elastic.co/u/iamredlus)\ **Post date:** [January 23, 2019, 11:48am UTC](https://discuss.elastic.co/t/percentiles-aggregation-does-not-behave-as-expected/164962/6 "2019-01-23T11:48:41Z")
Yep, I've experimented with different levels of the compression parameter (up to 2,000), but it does not seem to have any effect (maybe the sample size is small enough such that additional 'nodes' do not add accuracy). ---
**Author:** ![iamredlus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iamredlus/32/15504_2.png) [@iamredlus](https://discuss.elastic.co/u/iamredlus)\ **Post date:** [February 12, 2019, 3:22pm UTC](https://discuss.elastic.co/t/percentiles-aggregation-does-not-behave-as-expected/164962/7 "2019-02-12T15:22:01Z")
I've opened an issue in github on the subject - > ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 12, 2019, 3:22pm UTC](https://discuss.elastic.co/t/percentiles-aggregation-does-not-behave-as-expected/164962/8 "2019-03-12T15:22:07Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.