Wrong ML Job query packetbeat_rare_user_agent or missing event.dataset in network traffic data?

Elastic Security SIEM
1

Hello,

The query of the managed machine learning job packetbeat_rare_user_agent doesn't match with any documents. It seems like event.dataset is not in the logs-network_traffic.http-* datastream?

{"bool":{"filter":[{"term":{"agent.type":"packetbeat"}}],"should":[{"term":{"event.dataset":"http"}},{"term":{"event.dataset":"network_traffic.http"}}],"minimum_should_match":1,"must_not":[{"wildcard":{"user_agent.original":{"value":"Mozilla*"}}}]}}

Changed the to "event.dataset" to "data_stream.dataset" and it works.

Willem

2

@willemdh your assessment looks accurate, here: some of our newer integrations are populating data_stream.dataset instead of the earlier event.dataset, and our ML Jobs have not been updated to capture those data.

The team has filed an issue to track/update the ML job queries, but your workaround should also suffice in the interim: changing either the query or the data should address the issue.

1 Like
3

Thanks @RylandHerrick !