Threshold Alerts with Delayed Logs - Verification

[kbratt (Kent Bratt)]

I have set up Threshold Security Alerts that are running every 5 minutes and looking back an additional 1 minute..

We are working on fixing a performance issue where our Logs are delayed by up to 20 minutes at times.

Since our Logs can be delayed by up to 20 minutes, and my Threshold rules are running every 5 minutes, will my threshold rules not trigger during these times because by the time my logs are processed, they can be 20 minutes old already?

Thanks

[vitaliidm (Vitalii Dmyterko)]

Hey @kbratt
Welcome to Elastic community
If logs delayed by 20m, with rule's interval 5m + 1m lookback time(6m in total), these events won't be queried, if their timestamp is 20m in past.
There are 2 possible options:

1. Increase lookback time, so events with timestamp in past will be caught. In your case, if you anticipate 20m delay - 30m lookback time might be enough. But it won't guarantee that events will be processed, if delay would be 40m for example.
2. Use timestamp override option(preferable): Create a detection rule | Elastic Security Solution [8.11] | Elastic. See option k:

Timestamp override (optional): Select a source event timestamp field. When selected, the rule’s query uses the selected field, instead of the default @timestamp field, to search for alerts. This can help reduce missing alerts due to network or server outages. Specifically, if your ingest pipeline adds a timestamp when events are sent to Elasticsearch, this avoids missing alerts due to ingestion delays. However, if you know your data source has an inaccurate @timestamp value, it is recommended you select the Do not use @timestamp as a fallback timestamp field option to ignore the @timestamp field entirely.

Hope that will help to resolve the issue.
Thanks, Vitalii

[kbratt (Kent Bratt)]

Just what I needed, thanks @vitaliidm !