# Attack Discovery Questions and Feedback **URL:** \ **Category:** Elastic Security\ **Created:** [July 31, 2024, 7:30am UTC](https://discuss.elastic.co/t/attack-discovery-questions-and-feedback/364109 "2024-07-31T07:30:56Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\ **Post date:** [July 31, 2024, 7:30am UTC](https://discuss.elastic.co/t/attack-discovery-questions-and-feedback/364109/1 "2024-07-31T07:30:56Z")
Hello, So we updated to 8.14.3 earlier this week and I had the chance to play a bit with the Attack Discovery feature. It started by hitting the max content length of 8K tokens a lot. ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/c/0cd44dc8ab15f15aa7f16eff4731a07590f641e1.png) Asked our Azure engineer to provide a model with more tokens. In the meantime I decreased the Alerts knowledgebase from 30 to 10. The results were mixed imho. Most of the time we have between 250 and 400 alerts a day. How does it decide what alerts to analyse? I noticed it choose a lot of alerts triggering from the rule "Multiple Alerts in Different ATT&CK Tactics on a Single Host". After closing all these alerts they were still chosen by Attack Discovery. Other time I saw it choose 10 identical alerts. Imho it's not very useful when 10 exactly the same alerts are being analysed? Imho it would be nice if we could have some configuration options to help decide what alerts to analyse. For example only open alerts, only alerts in a specific time windows, only host or only network alerts, maybe even allow us to select the specific alerts to analyse? Best regards, WillemD ---
**Author:** ![Andrew\_G](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_g/32/49178_2.png) [@Andrew\_G](https://discuss.elastic.co/u/Andrew_G)\ **Post date:** [July 31, 2024, 5:29pm UTC](https://discuss.elastic.co/t/attack-discovery-questions-and-feedback/364109/2 "2024-07-31T17:29:17Z")
Thanks for your feedback @willemdh! > It started by hitting the max content length of 8K tokens a lot. > Asked our Azure engineer to provide a model with more tokens. In general, consider using one of the models in the `Excellent` category for Attack discovery in the [Large language model performance matrix](https://www.elastic.co/guide/en/security/current/llm-performance-matrix.html) documentation. The matrix is current as of `8.14.3`, but FYI `GPT-4o` Attack discovery output will be improved via this [PR](https://github.com/elastic/kibana/pull/187183) targeting an upcoming release. > In the meantime I decreased the Alerts knowledgebase from 30 to 10. The default of 20 alerts is typically under the token limits of the models in the [matrix](https://www.elastic.co/guide/en/security/current/llm-performance-matrix.html). The number of _input_ tokens consumed is influenced by: - The number of alerts sent as context (configured via the Knowledge base setting) - The fields sent in each alert (configured via the Anonymization setting) - The amount of data present in each field, which varies from alert-to-alert It's therefore possible to send less data, and less data per-alert via customized settings. However it's worth noting that the current generation of LLMs are also limited by _output_ tokens. Thus it's possible to reduce the size of the input by decreasing the number of alerts sent, or sending less data per alert, but reducing both may not be enough to overcome the output token limits of a model. We're experimenting with potential improvements to Attack discovery that work around some of the token limitations of the current generation of models. > Most of the time we have between 250 and 400 alerts a day. How does it decide what alerts to analyse? Attack discovery analyzes opened and acknowledged alerts from the last 24 hours. It includes up to `n` alerts (default: 20) as configured by the Knowledge Base [setting](https://www.elastic.co/guide/en/security/current/attack-discovery.html). The `n` alerts sent as context are sorted by risk score. Using the default of 20 as an example, the 20 alerts with the highest risk score in the last 24 hours will be sent as context. > I noticed it choose a lot of alerts triggering from the rule "Multiple Alerts in Different ATT&CK Tactics on a Single Host". After closing all these alerts they were still chosen by Attack Discovery. The global date selector on the Security \> Alerts page defaults to `Today`, which depending on when the alerts were generated, may include a different set of alerts than `Last 24 hours`. To approximate a view of the alerts sent as context for Attack discovery via the Alerts page: 1. Select `Last 24 hours` from the Security \> Alerts global date picker 2. Filter the page for open and acknowledged alerts 3. Sort the `Risk Score` column in the alerts table from `High` to `low` 4. Open the `Sort fields` popover on the alerts table, and drag `Risk score` above `@timestamp` to sort first by `Risk Score`, and then by `@timestamp` 5. Select `Rows per page: 20` (or higher) to match the configured number of alerts in the Knowledge base setting The above is illustrated by the screenshot below: ![alerts_page_filtering](https://us1.discourse-cdn.com/elastic/original/3X/8/6/86c54bc0669076e7998c201058c87081f5fbab7c.png) > Other time I saw it choose 10 identical alerts. Imho it's not very useful when 10 exactly the same alerts are being analysed? > > Imho it would be nice if we could have some configuration options to help decide what alerts to analyse. For example only open alerts, only alerts in a specific time windows, only host or only network alerts, maybe even allow us to select the specific alerts to analyse? Thank you for sharing this. We're exploring improvements to Attack discovery that may include additional filtering of the alerts that serve as input. ---
**Author:** ![jamesspi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jamesspi/32/24479_2.png) [@jamesspi](https://discuss.elastic.co/u/jamesspi)\ **Post date:** [July 31, 2024, 5:35pm UTC](https://discuss.elastic.co/t/attack-discovery-questions-and-feedback/364109/3 "2024-07-31T17:35:07Z")
Thanks for the response, @Andrew_G ! @willemdh I'll just add that that the standard token limits these days is 128k and beyond. You're actually paying more for GPT-4 at 8k, then GPT-4o at 128K. James ---
**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\ **Post date:** [August 1, 2024, 4:34pm UTC](https://discuss.elastic.co/t/attack-discovery-questions-and-feedback/364109/4 "2024-08-01T16:34:17Z")
Thanks (again) for all the info @Andrew_G and @jamesspi . I'll definitely test this and will experiment with other models asap. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 29, 2024, 4:35pm UTC](https://discuss.elastic.co/t/attack-discovery-questions-and-feedback/364109/5 "2024-08-29T16:35:06Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.