Transform kills the whole cluster

Hi I'm using elastic cloud (

I'm not sure if that's the correct category, but I want to run a transform for the past 6 month, and it's killing the cpu. the whole cluster becomes unresponsive (queries, kibana ..)

is there any option to tell the transform to run "slowly"..?

Unfortunately not yet, throttling will be part of 7.8, check the docs_per_second setting in the docs.

I can not give you a date when 7.8 will be available, but it should not be far off.

ok, i'll run it at night for now. thanks you guys for your hard work

It seems to me that your cluster has some general performance problems or is simply under-powered with this task.

To further analyze this problem it seems better to ask support.

Logging queries that transform does, is a good approach. You can take the query from the log and execute it in dev console and see how fast it performs there. I guess it will be slow, too.

Transform can only go as fast as aggregations + bulk indexing.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.