Filter outliers elastic 5


(liorg2) #1

whats is the best way to do the following query in elastic 5.x:

find the average of a field, after removing the top 5% highest values (> 95th percentile) ?

I'm looking a single query solution

thanks!


(Isabel Drost-Fromm) #2

Can you elaborate on what's your reason for doing this in one query?

If that's not a hard requirement, you could be using percentile aggregations/ average aggregations in separate queries.

Isabel


(liorg2) #3

My purpose is to remove outliers, and get the avarage of values, lower than 90th percentile or so,

This aggregation is a child of several parent terms aggregations.

If i have to run 2 queries for the purpose above, I will need to run another query for each dimension combination.

For example:
In case of parent terms aggregations of the following dimensions: city, country, page device,
If we assume 1000 combinations,
After getting the 95th percentile value from the first query, I will need to run another query with filter of the above dimensions and also filter of value < 95 percentile X 1000 options


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.