We have a watcher running for a daily ML job (1d bucket span) over .ml-anomalies* and fires for record results where in_interim is false and some other conditions on record_score and actual values. The watcher has a schedule interval of 30h and filters results:
} } }
The watcher fires with some of the executions, however we still see some executions which are not firing while there are results matching in ml-anomalies. If we simulate watcher execution and change the timestamp to cover those results we can see the watcher is firing and showing the results, so the query and overall script conditions looks fine for those runs. I have seen this articles recommending to have an interval twice of the bucket-span. However, I'm still wondering if any other part of the configurations (ML job or watcher) can cause these missing results? What are the best approaches for ML watcher configurations so we can be sure not missing results?