Filter out results from aggregation based on info from multiple documents

Hi,

Here's a simple version of what I'm trying to do:

I have a bunch of documents with fields 'id' and 'status'. There can be many documents with the same 'id' value. The value of 'status' can either be 'success' or 'fail'.

I'm trying to create a horizontal bar chart visualization
First, I aggregate documents by unique id, like this:

Then I split the series by 'status' value, getting the following result:
(each bar corresponds to each unique 'id' value)

image

TL;DR: I want to get rid of all only-red bars and keep those which have some green in them

Longer version: Generally speaking, each unique id corresponds to an opportunity and each document with this 'id' value represents an attempt to take advantage of that opportunity. Most opportunities result in complete failure - where each attempt fails, so each doc with that 'id' has 'fail' in their 'status' field. What I'm interested in are opportunities where at least some attempts succeed. Ideally I'd like to filter based on percentage of successes, but being able to keep only aggregations which have at least one success in them is also acceptable.

The problem is that I need to make a decision to filter or keep an aggregation based on data from multiple docs. Is there a way to do that in Kibana/Elasticsearch? Alternatively, I could go over every index matching the index pattern, calculate the 'success_percentage' value for each unique id and append that value to each doc as a new field - is that possible?

I'm using Elastic Stack version 7.5.1

Update: I realized that I can filter out all docs where 'status' is 'fail' and thus be left with only aggregations containing successes. However, that way I can't see the success to fail ratio.

Hi @SiriKeet,

If I am understanding your question correctly, this sounds like a use case for bucket selector aggregations, since it sounds like you want to exclude certain buckets based on the results of their aggregated counts.

Unfortunately, these are not yet supported in Kibana, although you can follow along with the relevant Github issues here and here to be notified of any updates.

And for future reference, there's also this issue which tracks ES aggregations support across Kibana.

Hope this helps a bit -

Thanks,

Luke