Recommended practise for detection tuning; filters or exceptions

Elastic Security SIEM
1

Hi,

I have given a look through the documentation but did not come across anything conclusive.
What is the recommended way of tuning detection rules, as you can get the same results from filters and exceptions.

Can you also explain why that is the recommended way, trying to settle an internal discussion :slight_smile:

Thanks in advance.

2

Hi @The1WhoPrtNocks, thank for the great question!

As you've discovered, since Elastic Security/SIEM is built upon the Elastic Stack, a very flexible search and analytics engine, there are often more than one way to accomplish a given task.

In your case, tuning detections rules can indeed be accomplished several ways, including:

  • modifying the rule detection logic (e.g., rule query)
  • adding filters to the rule query (similar to above)
  • adding exceptions to the rule

From a design philosophy point of view, if the modification affects the logic or theory of operation of the detection rule, it might fit best as a modification to the rule query or a filter. However if the modification does not change the logic of the rule, but is simply adding an environment-specific condition where you choose not to apply the rule, then an exception will likely be the best fit.

From a practical point of view, rule exceptions are more flexible - they support lists, which can be centrally maintained, and in the future, may be able to be shared across multiple rules. So our general advice is to use exceptions for tuning rules.

We've provided some specific tuning guidance in this doc section:

Hope this helps!

2 Likes
3

My personal consideration between a filter and an exclusion is if I want the signal to be logged or not. Use a filter and you will not get a signal, use an exception and you will get a signal which gets autoclosed. Huge difference imho.

4

Thanks to both for your answers, exactly what I was looking for :slight_smile:

5

Use a filter and you will not get a signal, use an exception and you will get a signal which gets autoclosed. Huge difference imho.

Thanks @willemdh but I think that's not quite right.

Both methods (filters and exceptions) will cause no further alerts (signal documents) to be generated when their respective conditions are met.

What may be causing confusion is that the exception dialog allows for the optional closing of existing alerts (that may have existed before the exception was created).

image
image1644×870 148 KB

Hope this helps,
Mike P.

6

Oh, thanks Mike, seems I assumed wrong indeed. Thanks for the clarification..

Just a thought, imho my (wrong) assumption would be a nice feature no? This would kind of allow stuff to be detected, but autoclosed for whatever reason.

7

Just a thought, imho my (wrong) assumption would be a nice feature no? This would kind of allow stuff to be detected, but autoclosed for whatever reason.

Good idea @willemdh we agree that there is a detection benefit by having alerts created by rules, but that are not presented in the "open" alerts list. However, we felt that putting them in the "closed" state might incorrectly imply that an analyst took action to close them, and might not offer the granularity of alert state that some SOC's might require.

So instead of auto-close, we provide the ability to mark a rule as a "building block" rule during the rule creation dialog (step 2 advanced settings), which causes alerts to be generated when the rule conditions are met, but these alerts will not show up in the Detection alerts table under any status {open, closed, in progress}. Some detection engineers find this capability useful when creating more advanced "2nd order" rules that run on previously detected alerts!

image
image681×509 38.1 KB

If the analyst really wants to see the "building block" alerts, there's a dedicated enable button on the right hand side of the Detection alerts table, shown below.

image

Thanks for your continued feedback!

8

Personally I haven't used the building block alerts and I'm not such a big fan, as imho it increases complexity (while we are already overworked) and also applies for the whole rule. If the closed state would not be appropriate, maybe a 4th state type could introduced 'archived' or sth similar. The reason I (personally) still think it would be a nice feature to allow rules to generate signals, but auto-close / auto-archive them would to be for example add more temporary 'exclusions', for example for external or temporary employees for which we know they will access / test / penetrate stuff for a limited time. If after the facts, an investigation would be required or suspicion has been raised that someone might have accessed something they shouldn't we can analyse the closed/archived signals after the facts.

Anyway, just shooting some thoughts :slight_smile:

(Thanks a lot for this amazing product)

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.