We have just recently started to adopt Elastic Defend and really impressed by it. One thing I found inconvenient is that the output alerts don't mention the fleet policy or datastream namespace. We have our fleet segmented over a number of policies and would also want to keep this segmentation in the rules. Any quick solutions to fix this?
I believe you are correct about the Elastic Defend alerts not containing the associated policy name in the alert. However, as far as I know the namespace is present in the alerts. Would this work for your purposes?
Example from a defend alert:
data_stream": {
"namespace": "default",
"type": "logs",
"dataset": "endpoint.alerts"
}
@catn0b0t it sounds like you may be describing this behavior, which we are planning to address in a future release. If so, please that issue and/or share your use case there!
FYI there are currently workarounds involving runtime fields, which are mentioned there as well.
Assuming you're talking about Defend/Endpoint alerts (those originally generated on the host running Agent and visible under the Elastic Security Detection Engine rule) it looks like 8.18 will have the policy info you're looking for if you're running an 8.18.0+ Agent.
That work I linked to will only affect Defend alerts not alerts originally generated by Detection Engine rules because policy details aren't being added to most Defend event documents.
Oh so it is coming if I understand correctly? Cool!
Yes, I figured that there will probably be a way to fix this in the pipeline by looking up the parent document from which the alert generated.
By thinking about the question I posted here I also figured that my question isn't as straight forward as it might seem at first glance. For example, consider the rules that alert for lateral movement: it might perfectly make sense that the originating events stem from several different policies in different namespaces, that is the nature of a lateral movement attack. But in that case it is not clear what the policy/namespace on the alert should actually be.
Defend alerts containing some policy information is coming.
From your last post though it seems like you're also interested in Detection Engine (SIEM) rules (great!). Only alerts that are generated by Elastic Defend integration using its Malware, Ransomware, Memory Threat, or Malicious Behavior protections will contain the policy information. Those alerts are generated on the host running Elastic Agent by the Elastic Endpoint binary.
Other Detection alerts are generated in Kibana based on source data (for instance, Defend process events or audibeat events). Adding policy information to all of those raw events would have a much larger effect on overall data volume.
Does that make sense? It seems like you had an idea how to use pipelines?