Hi
I have upgraded to 7.11.0 and now 7.11.1 and I think there is an issue with Threshold detections.
I have a simple detection based on my FW logs. It looks for flow_denied or flow_dropped messages and raises a detection if a single source.ip has >= 500 events.
So the query is straight forward and the group by field is source.ip.
This used to work fine and I have a timeline template which was showing me the expected results and working well.
The rule now seems to fire for almost anything and is missing many of the fields. Many of the results that are returned don't even have a source.ip field.
The query is correct, I can apply the same filter and build a visualisation and it shows me the results as expected.
I'm no expert on this so may be wrong here...however, I have viewed my detection rule, everything is in place and I click the preview which shows it would be very noisy, as shown below. If I then click the Inspect button related to that preview is shows me the query that was sent, also shown below.
I don't fully follow the query language, however I can't see anything in there that references the grouping option of source.ip and a threshold of 500 events. At a minimum, I would expect to be able to search that block of text and find source.ip in there but it's not.
As I say, possible my misunderstanding but if this is not the issue, there does seem to have been a problem introduced with threshold detections since 7.10.2.
I have also tried creating a new rule in case the old one failed to migrate from 7.10.2 to 7.11.x for some reason but the result is the same.
Thanks in advance
Phil
{
"aggregations": {
"eventActionGroup": {
"terms": {
"field": "event.category",
"missing": "All others",
"order": {
"_count": "desc"
},
"size": 10
},
"aggs": {
"events": {
"date_histogram": {
"field": "@timestamp",
"fixed_interval": "112500ms",
"min_doc_count": 0,
"extended_bounds": {
"min": 1613742526822,
"max": 1613746126822
}
}
}
}
}
},
"query": {
"bool": {
"filter": [
{
"bool": {
"must": [],
"filter": [
{
"bool": {
"filter": [
{
"bool": {
"should": [
{
"match": {
"event.module": "panw"
}
}
],
"minimum_should_match": 1
}
},
{
"bool": {
"should": [
{
"bool": {
"should": [
{
"match": {
"event.action": "flow_denied"
}
}
],
"minimum_should_match": 1
}
},
{
"bool": {
"should": [
{
"match": {
"event.action": "flow_dropped"
}
}
],
"minimum_should_match": 1
}
}
],
"minimum_should_match": 1
}
}
]
}
}
],
"should": [],
"must_not": []
}
},
{
"range": {
"@timestamp": {
"gte": "2021-02-19T13:48:46.822Z",
"lte": "2021-02-19T14:48:46.822Z",
"format": "strict_date_optional_time"
}
}
}
]
}
},
"size": 0,
"track_total_hits": true
}
