Setting Varying Non-Relative Watcher Timeranges?

[Kenneth_M_Kolano]

I'm hoping to set up a Watcher to monitor an index and provide a report when results appear. I initially figured I could use the "interval" schedule along with a search with some timestamp checks...

"@timestamp": {
     "gte": "now-5m"
}

...but I have a feeling that such a setup is likely to sometimes drop out results that occur near the edges of each 5m interval.

Is there any mechanism to allow specify more exacting time intervals to avoid the possibility of there being short uncovered gaps between intervals?

I have a feeling this may be even more critical now that nanosecond times are supported.

[spinscale (Alexander Reelsen)]

Yes, this is indeed a problem around the edges. You could look at the last execution of the watch (by looking at the watch history) and then use the last execution as the lower boundary for your query.

However this still won't fully solve your problem. If your ingestion pipeline is slower than the specified interval (like 5m in this case), you could still be missing documents. In order to prevent that, you would actually need to mark each document, which is not feasible at for a logging use-case. You could also have overlapping query intervals and then deal with potentially duplicate alerts instead of missing them.

It's a lot of tradeoffs here and you will need to pick some.