Aggregate a field ignoring the active filters for the same field

[Boo]

Hello,
I am new with elasticsearch and I am trying to make a aggregation to create options for a muliple selection.

For example my documents have a list of nested documents. The nested documents have some attribute (for example 'size' and 'color'). I want to aggregate all possible variants to make multi select filter.

At the moment my aggregation looks like this:

{
    "aggs" : {
        "variants" : {
            "nested" : {
                "path" : "variants"
            },
            "aggs" : {
                "color" : { "terms" : { "field" : "resellers.color" } },
                "size" : { "terms" : { "field" : "resellers.size" } }
            }
        }
    }
}

(The snippet may not be correct because it is weekend and I have no access to the data from my home)

Now I want the aggregation of a field to avoid all filters for the same field.
So when I filter for "color": "red" and "size":"xl" the aggregation should ignore the "color" filter but note the "size" filter to create the "color" options. On the same run the "size" aggregation have to ignore the "size" filter and notethe "color" filter.

My question is: Is it possible to aggregate all fields correctly in one request? Or do I have to make for every single aggregation an own request?

I hope I could explain my request understandably.
(English is not my native language and I am bad in explaining my problems ... even in my native language)

Thanks
Lukas