Kibana Machine Learning Job Alert

Elastic Stack Kibana
1

Hi Team,

We have created alert for anomaly detection and we are getting this alert on email.
It is showing different timestamp

Elastic Stack Machine Learning Alert:
- Job IDs: {{context.jobIds}}
- Time: {{context.timestampIso8601}}
- Anomaly score: {{context.score}}

{{context.message}}
{{#context.topInfluencers.length}}
  Top influencers:
  {{#context.topInfluencers}}
    {{influencer_field_name}} = {{influencer_field_value}} [{{score}}]
  {{/context.topInfluencers}}
{{/context.topInfluencers.length}}

{{#context.topRecords.length}}
  Top records:
  {{#context.topRecords}}
    {{function}}({{field_name}}) {{by_field_value}} {{over_field_value}} {{partition_field_value}} [{{score}}]
  {{/context.topRecords}}
{{/context.topRecords.length}}

{{! Replace kibanaBaseUrl if not configured in Kibana }}
[Open in Anomaly Explorer]({{{kibanaBaseUrl}}}{{{context.anomalyExplorerUrl}}})

Result in the email

Elastic Stack Machine Learning Alert:
•	Job IDs: agg-pss-ist-awesant
•	Time: 2023-07-24T10:00:00.000Z
•	Anomaly score: 85

Alerts are raised based on real-time scores. Remember that scores may be adjusted over time as data continues to be analyzed.

Top influencers: PCode.keyword = 011000 [97] Destination.keyword = 549627 [97] Source.keyword = 0093 [97]

Top records: max(hits) [85]

Anomaly was detected at 15:30-15:35 but it is showing different time
i have shared screenshot for this . I would like to know in the screenshot it is showing July 24th 2023, 15:00 (highlighted part) but in the table it is showing actual timerange. Can you please tell me also on this part

ml2
ml2954×741 53.7 KB

2

Hi @Aniket_Pant ,

Could you please share complete configuration details of your anomaly detection job and alerting rule as well? There are several important factors define the alerting context, e.g. what result type is configured in alerting rule, lookback interval, etc. If it's a bucket result, then you receive a timestamp of beginning of the anomalous bucket. The timestamp on the screenshot you shared belongs to the anomaly record. It's the most anomalous records within the lookback interval you configured in your alerting rule.

You can read more about alerting configuration for anomaly detection jobs in this blog post. Hope it helps.

3
Summary

This text will be hidden

{
  "job_id": "some_job_id",
  "job_type": "anomaly_detector",
  "job_version": "7.17.5",
  "create_time": 1689751050137,
  
    },
    "query": {
      "bool": {
        "filter": [
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "status.keyword": "value"
                  }
                },
                {
                  "term": {
                    "status_1.keyword": "value"
                  }
                },
                {
                  "term": {
                    "aggrType.keyword": "5m"
                  }
                }
"description": "detecting anomaly for business decline",
  "analysis_config": {
    "bucket_span": "5m",
    "detectors": [
      {
        "detector_description": "max(hits)",
        "function": "max",
        "field_name": "hits",
        "detector_index": 0
      }
    ],
    "influencers": [
      "Source.keyword",
      "Destination.keyword",
      "PCode.keyword"
    ]
  },
  "analysis_limits": {
    "model_memory_limit": "11mb",
    "categorization_examples_limit": 4
  },
  "data_description": {
    "time_field": "@timestamp",
    "time_format": "epoch_ms"
  },
 "alerting_rules": [
    {
      "id": "8c13cf80-26c7-11ee-8bf5-9f9a21848a63",
      "notifyWhen": "onActionGroupChange",
      "consumer": "alerts",
      "tags": [
        "aggr-pss-ist-awesant",
        "anomaly detection"
      ],
      "name": "Business Decline Alert",
      "enabled": true,
      "throttle": null,
      "alertTypeId": "xpack.ml.anomaly_detection_alert",
      "apiKeyOwner": "lc5684231",
      "createdBy": "lc5684231",
      "updatedBy": "lc5684231",
      "muteAll": false,
      "mutedInstanceIds": [],
      "schedule": {
        "interval": "5m"
      },
      "actions": [
        {
          "group": "anomaly_score_match",
          "params": {
            "documents": [
              {
                "message": "{{context.metric}}",
                "rule_id": "{{rule.id}}",
                "reason": "{{context.reason}}",
                "timestamp": "{{context.timestamp}}"
              }
            ]
          },
          "actionTypeId": ".index",
          "id": "0ded0760-26c3-11ee-8bf5-9f9a21848a63"
        },
        {
          "group": "anomaly_score_match",
          "params": {
            "message": """Elastic Stack Machine Learning Alert:
- Job IDs: {{context.jobIds}}
- Time: {{context.timestampIso8601}}
- Anomaly score: {{context.score}}

{{context.message}}

{{#context.topInfluencers.length}}
  Top influencers:
  {{#context.topInfluencers}}
    {{influencer_field_name}} = {{influencer_field_value}} [{{score}}]
  {{/context.topInfluencers}}
{{/context.topInfluencers.length}}

{{#context.topRecords.length}}
  Top records:
  {{#context.topRecords}}
    {{function}}({{field_name}}) {{by_field_value}} {{over_field_value}} {{partition_field_value}} [{{score}}]
  {{/context.topRecords}}
{{/context.topRecords.length}}

{{! Replace kibanaBaseUrl if not configured in Kibana }}
[Open in Anomaly Explorer]({{{kibanaBaseUrl}}}{{{context.anomalyExplorerUrl}}})
""",
            "to": [
              "xx.com",
              "xx.com"
            ],
            "subject": "Anomaly Detection For Business Decline"
          },
          "actionTypeId": ".email",

Hi @darnautov

4

Hi Team,

Can you please me in easier terms that if anomaly was detected at 00:10 AM - 00:15 AM. But in Single metric Explore If we Anomaly detection table in time field column it shows 00:00 please see the below screenshot. It has to show when anomaly was detected. Please see the highlighted part

Screenshot-5
Screenshot-51915×750 64.6 KB

5

Hi Team,

Can you please help me

6

Click on the dropdown box labeled "Interval" and select "Show all".

When set to Auto, the table will collapse anomalies that appear in consecutive buckets into a single row.

7

Hi @richcollier ,
With "Show all" it is showing correct timestamp

When set to Auto, the table will collapse anomalies that appear in consecutive buckets into a single row

Could you please explain me in a simple terms

8

If set to Auto, it will aggregate the anomalies and show a summary in the table (it will aggregate them based on time and offending entity) - showing the max anomaly score for each interval/entity.

See another post about this topic.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.