ES Continuous Transforms Checkpoint not updating

For running transform in continuous mode the sync section is mandatory. It's basically used to find out what has changed. The idea is to only do the necessary updates but not a full re-run.

sync.time.field must be a date field that is using a real timestamp. A checkpoint is basically a timestamp until all data has been processed and the next time transform triggers it looks for new data that has arrived after the checkpoint. For the same reason transform will not be able to see new documents with a historic timestamp. If you don't know what to take as field, you can use a ingest pipeline and add an ingest timestamp as described here.

sync.time.delay describes how much time transform should reserve for ingest processing and data coming in out of order. For example: Assume the timestamps are created on some IoT device and only send once every minute as batch. Network takes some millisecondes, next you have a queue, than ingest, finally it enters elasticsearch. Note the index refresh interval which adds another second. Delay is basically the sum of the steps I described, with other words, when transform sees the new datapoint, it can be worst case sum(steps) ms old. That's your ingest delay. There is a trick: If you use the suggested ingest timestamp, you skip a lot of the steps above and your worst case assumption becomes a lot easier.

I hope that helps.