So our ingestion mechanism for ingesting data had stopped for an hour, now my machine learning jobs refuse to pull in any new documents. I have verified it can see new latest document dates but just doesn't pull them in or do any analysis them. It also alerting that it has seen any new documents when the ingestion stopped, but now still won't take any of the new documents that I know for a fact are getting ingested and showing up in search. How do I make the ML jobs start taking the documents in? I already restarted the data-feed.
The ML job, by default, will not go "back in time" to re-process bucket_spans that have already been analyzed.
You can, however, figure out the last model snapshot taken before your ingest outage, revert to that snapshot, and delete the intervening results.
Then, just restart the datafeed from the point of the time of the snapshot and continue live.
This is possible via the UI in recent versions (I think v7.11+) from the Jobs Management page
So my question is why does the datafeed just stop loading any future data it gets on time? do I need to delay my data query delay far enough back?
anotherwords it just stops loading data altogether if it hits a period of no data.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.