Why is the "Alert Delay of 5 Consecutive Matches" Not Being Met?

Here's the English paragraph with "CUSTOM_ENTIDAD.keyword : "Redipro" and" removed from the query filter:


Thread Title: Why is the "Alert Delay of 5 Consecutive Matches" Not Being Met for My Kibana Threshold Rule?

Hello everyone,

I'm using Kibana 8.18.3 and have a threshold alert rule configured with the following characteristics:

  • Data View: prc-chi-logs-sia-informe-operaciones-v2*
  • Query Filter: Operacion.keyword : "UPDATE" and COD_CANAL_T.keyword : "CANAL 1 ISO-2 ATM" or Operacion.keyword : "UPDATE" and COD_CANAL_T.keyword : "CANAL 3 ISO-2 POS"
  • Condition: COUNT all documents IS BELOW OR EQUALS 3
  • Group alerts by: CUSTOM_ENTIDAD.keyword and COD_CANAL_T.keyword
  • Rule Schedule: Every 1 minute
  • Alert Delay: Alert after 5 consecutive matches

Every time an alert is triggered for a group like CUSTOM_ENTIDAD: [Example Entity], COD_CANAL_T: "CANAL 1 ISO-2 ATM" (for example, with a count of 2 at 12:27 PM), I observe the same symptom: the "Alert Delay" of 5 consecutive matches does not seem to be respected. Upon reviewing the historical data in Kibana Discover for the exact group that alerted, I find that the minutes leading up to the alert do not meet the condition. For instance, in the example mentioned, the 4 preceding minutes (12:23 PM - 12:26 PM) had consistently above 3 counts, and only in the alert minute (12:27 PM) did the count drop to 0, being the first time it met the condition in that sequence.

My expectation is that the alert should not have been triggered in these cases, as the history does not show the 5 consecutive matches required.

Has anyone else experienced this recurrent behavior or has any idea why this might be happening? Any guidance would be greatly appreciated.

Thank you in advance!

PUT kbn:/api/alerting/rule/a6d1f927-ff5a-4c98-9a5d-2cfd80f6f657
{
  "name": "OBS | ALERTA CAIDA CANAL REDIPRO [QA]",
  "tags": [
    "OBS",
    "REDIPRO",
    "QA"
  ],
  "schedule": {
    "interval": "1m"
  },
  "params": {
    "criteria": [
      {
        "comparator": "<=",
        "metrics": [
          {
            "name": "A",
            "filter": "CUSTOM_ENTIDAD.keyword : \"Redipro\"  and Operacion.keyword : \"UPDATE\" and COD_CANAL_T.keyword : \"CANAL 1 ISO-2 ATM\"  or CUSTOM_ENTIDAD.keyword : \"Redipro\"  and Operacion.keyword : \"UPDATE\" and COD_CANAL_T.keyword : \"CANAL 3 ISO-2 POS\" ",
            "aggType": "count"
          }
        ],
        "threshold": [
          3
        ],
        "timeSize": 5,
        "timeUnit": "m"
      }
    ],
    "alertOnNoData": false,
    "alertOnGroupDisappear": false,
    "searchConfiguration": {
      "query": {
        "query": "CUSTOM_ENTIDAD.keyword : \"Redipro\"  and Operacion.keyword : \"UPDATE\" and COD_CANAL_T.keyword : \"CANAL 1 ISO-2 ATM\"  or CUSTOM_ENTIDAD.keyword : \"Redipro\"  and Operacion.keyword : \"UPDATE\" and COD_CANAL_T.keyword : \"CANAL 3 ISO-2 POS\" ",
        "language": "kuery"
      },
      "index": "a100e832-8599-4b8b-926a-8c5616ad0d5c"
    },
    "groupBy": [
      "CUSTOM_ENTIDAD.keyword",
      "COD_CANAL.keyword",
      "COD_CANAL_T.keyword"
    ]
  },
  "actions": [
    {
      "group": "custom_threshold.fired",
      "id": "566c725e-555c-42c4-b642-5c1b0fca0a74",
      "params": {
        "message": "- **Nombre:** {{rule.name}}\n- **ID Alerta:** {{rule.id}}\n- **Cauza Raiz:** {{alert.id}}\n- **Detalle Alerta:** {{context.reason}}\n- **URL Alerta:**\n[View alert details]({{context.alertDetailsUrl}})\n\n- **Estado:** ALERTA ROJA\n\n<img src=\"https://img.freepik.com/vector-gratis/circulo-mensaje-advertencia_78370-4053.jpg?t=st=1735930141~exp=1735933741~hmac=2bc2f811968f41c327eaa87b0d01faf82641000b94d377435ca26bb47f8eea3e&w=150\" />"
      },
      "frequency": {
        "notify_when": "onActionGroupChange",
        "throttle": null,
        "summary": false
      },
      "alerts_filter": {
        "timeframe": {
          "days": [
            7,
            6,
            5,
            4,
            3,
            2,
            1
          ],
          "timezone": "America/Santiago",
          "hours": {
            "start": "06:00",
            "end": "22:00"
          }
        }
      },
      "uuid": "746ad726-92c5-4b6d-8e3a-7034cb01c65a"
    },
    {
      "group": "recovered",
      "id": "566c725e-555c-42c4-b642-5c1b0fca0a74",
      "params": {
        "message": "- **Nombre:** {{rule.name}}\n- **ID Alerta:** {{rule.id}}\n- **Cauza Raiz:** {{alert.id}}\n- **URL Alerta:**\n[View alert details]({{context.alertDetailsUrl}})\n- **Duracion:** {{context.alert.duration.us}}\n- **Estado:** RECUPERACION\n\n<img src=\"https://img.freepik.com/vector-premium/ok-icono-senal-verde-web-aplicaciones-signo-marca-verificacion-ilustracion-stock-vectorial_100456-5938.jpg?w=150\" />"
      },
      "frequency": {
        "notify_when": "onActionGroupChange",
        "throttle": null,
        "summary": false
      },
      "alerts_filter": {
        "timeframe": {
          "days": [
            7,
            1,
            2,
            3,
            4,
            5,
            6
          ],
          "timezone": "America/Santiago",
          "hours": {
            "start": "06:00",
            "end": "22:00"
          }
        }
      },
      "uuid": "9ebc05e9-d957-498f-8cce-79065596aabd"
    }
  ],
  "alert_delay": {
    "active": 5
  }
}
1 Like

Hello @Diego_Ramirez

Welcome to the community.

  1. Looking at your query try to use below with brackets as without brackets it might evaluate and followed by or :

(Operacion.keyword : "UPDATE" and COD_CANAL_T.keyword : "CANAL 1 ISO-2 ATM") or (Operacion.keyword : "UPDATE" and COD_CANAL_T.keyword : "CANAL 3 ISO-2 POS")
  1. You have below fields populated for all documents, right?
    CUSTOM_ENTIDAD.keyword, COD_CANAL_T.keyword

  2. You have checked the data count as per below logic, right ?

Rule execution at Time Consideration (from => to) Status
12:22 12:17 to 12:22 did not match
12:23 12:18 to 12:23 matched (count <=3 )
12:24 12:19 to 12:24 matched (count <=3 )
12:25 12:20 to 12:25 matched (count <=3 )
12:26 12:21 to 12:26 matched (count <=3 )
12:27 12:22 to 12:27 Alert

Thanks!!