Hello,
I have a question regarding scroll context-s created by the Update by query API.
Currently we create a lot of scroll context, when we update events in our system. The scrolls are kept for just one minute, but due to the bursty nature of our update process we are hitting the scroll limits of the cluster.
Those bursts of scrolls happen just a few times during the day, so the overall resource consumed by the scrolls does not impact the system that much(only in those windows, but I could be wrong).
Currently we changed the scroll context limits to 2048, which is a temporary quick solution.
As there is a way to delete the scrolls manually, in theory we should be able to delete the unnecessary scrolls after the update, the problem is that using _update_by_query does not return any information about the scroll, that it uses, when doing the updates.
Is there a way to find the scroll used by _update_by_query, as I was not able to find one.
This way we will be able to free those resources, once the update has finished and not wait for elastic to clean them internally.
Thanks in advance.