Transforms - enabling filters as group_by

Right now options for grouping by in transforms are terms, histogram and date_histogram.
Is the plan adding filters anytime soon?
If not, I guess best option would be to create different transforms (one for each filter) for different indices, each pointing to a common alias?

No plans yet, the 1st step would be to add support to composite aggregations.

I wonder, do you have complicated filters in mind? Maybe you could achieve your usecase with a scripted group_by?

Can you provide an example what you try to do?

Sure. I am grouping requests by user_agent, but I'd like to reduce cardinality, since there are hundreds of different agents, into a shorter list like "iPhone", "Android", "Googlebot", "Desktop", etc. Separating using regex expressions.
I don't understand how to use this scripted group_by you mention :thinking:

Here is an example which works on user agents.

There is a problem however, script break continuous transform, see this issue. Currently transform allows you to use continuous, however it won't work correctly. Scripts are hard to handle, as I can not make any assumptions about the output.

If you want to do something like this in continuous mode, it's better to fix the data during ingest.

Yes... I do like solution B. Until then I guess I'll consider modifying ingest.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.