I'm trying to configure my ML Job + Watcher in a specific ingestion scenario:
- Ingestions occurs once in a day and the events timestamps are aggregated in the beginning of the day.
- The daily ingestion updates the day before yesterday (now-2d/d) values, but also inserts some documents in yesterday bucket (now-1d/d) - that will be replaced in the next ingestion (tomorrow).
- The daily ingestion also update old buckets (e.g.: last week buckets).
- It should ignore last bucket (now-1d/d) since this value will be replaced tomorrow;
- It should reanalyse old buckets updates (e.g: last week buckets).
- The watcher should target the day before yesterday (now-2d/d) with the correct value analysis (I think this can be accomplished with a
ltefilter in watcher)
I tested several
delay values for datafeed but it seems that the job always fetch data from last bucket before it updates to the correct value (that's why I need to ignore last datapoint):
Is it possible to configure a Job with these rules above?