Hi,
Have you considered using continuous transform instead of a batch transform?
From docs:
You can decide whether you want the transform to run once or continuously. A batch transform is a single operation that has a single checkpoint. Continuous transforms continually increment and process checkpoints as new source data is ingested.
Docs: Transform overview | Elasticsearch Guide [8.11] | Elastic