Detection Rule - Output of a aggregation bucket should match with other types of logs in the same index

Continuing the discussion from Detection Rule - Output of a aggregation bucket should match with other types of logs in the same index:

Hi @jancodenew thanks for using Elastic Security for SIEM!

If I'm understanding your question correctly, you want to detect when a MAC address that has been previously denoted as "rogue" is subsequently seen without the "rogue" notation, by looking at data in the same index. Is that correct?

Assuming such a use case, there's not currently an easy way to make that detection using Elastic Security. However, you may be able to accomplish a similar detection using a combination of two "building block" detection rules and one additional rule to tie the results together, so a total of 3 detection rules. Here's an explanation of what I mean.

First, we create two "building block" rules. (Building block rules are just like regular rules, except that their alerts do not show up in the Alerts table, avoiding noise). There's a checkbox in the rule's advanced settings to make any rule a "building block" rule.

The first building block rule is a threshold rule that creates an alert for every MAC address that has the "rogue" notation. (Note that I am using ECS-compliant fields in my example - you should normalize your data to ECS). Also note that my data does not have a field for the "rogue" notation, so I am substituting process.name:"Google Chrome Helper" as my "rogue" indication.

The second building block rule is also a threshold rule that creates an alert for every MAC address that has common.role field but does not contain the "rogue" value. Again, since my data does not have a field for the "rogue" notation, I am substituting not process.name:"Google Chrome Helper" as my not "rogue" indication.

The third rule is an event correlation rule that looks for sequences of alerts that have been generated by the two building block rules above joined by a common mac address using the host.mac field. Note that this rule is correlating based on the names of the building block rules, since the "rogue" indicator field is not present in the alert documents. The EQL sequence simply looks for any MAC address for which we've seen a "rogue" alert followed by a "non-rogue" alert.

I have set these rules to run every 5 minutes just to illustrate the example, but you will want to tune the rule intervals to meet your requirements.

Once all three rules are running, the alerts table will show only the alerts from Rule 3of3, and will indicate that we have 5 MAC addresses which have switched from Rogue to Non-Rogue.

Please let us know if this helps, or if you have any other questions.