How to have the last timestamp field in Kibana Alerts

Elastic Stack Kibana
1

Hi,

I have a use case in which Kibana alerting sends the alert when no documents are present in last 10 minutes.

How can i add the timestamp of when the last document was found in message body?

2

Hello @mosaadshaikh1998

  1. Using single rule it might not be possible as we check last 10 minutes & if there are no hits , means no document fetched hence cannot find the last timestamp for the document. If we increase the time duration from 10 minutes to 24 hours , we will be able to find the last timestamp but no tell if for last 10 minutes there was data received or not.

2. If you are OK ,both the conditions can be combined using a Watcher something like below or you will have to create 2 separate rules.

{
  "trigger": {
    "schedule": {
      "interval": "5m"
    }
  },
  "input": {
    "chain": {
      "inputs": [
        {
          "recent_data": {
            "search": {
              "request": {
                "search_type": "query_then_fetch",
                "indices": [
                  "kibana_sample_data_logs*"
                ],
                "rest_total_hits_as_int": true,
                "body": {
                  "query": {
                    "range": {
                      "@timestamp": {
                        "gte": "now-10m"
                      }
                    }
                  }
                }
              }
            }
          }
        },
        {
          "last_record": {
            "search": {
              "request": {
                "search_type": "query_then_fetch",
                "indices": [
                  "kibana_sample_data_logs*"
                ],
                "rest_total_hits_as_int": true,
                "body": {
                  "size": 1,
                  "sort": [
                    {
                      "@timestamp": {
                        "order": "desc"
                      }
                    }
                  ],
                  "_source": [
                    "@timestamp"
                  ]
                }
              }
            }
          }
        }
      ]
    }
  },
  "condition": {
    "script": {
      "source": "return ctx.payload.recent_data.hits.total == 0;",
      "lang": "painless"
    }
  },
  "actions": {
    "send_email": {
      "throttle_period_in_millis": 3600000,
      "email": {
        "profile": "standard",
        "to": [
          "myemail@com"
        ],
        "subject": "No data received in the last 10 minutes",
        "body": {
          "text": """No documents were found in the last 10 minutes.
Last record was received at: {{ctx.payload.last_record.hits.hits.0._source['@timestamp']}}"""
        }
      }
    }
  }
}

Thanks!!

3

Thank you for your prompt response @Tortoise

I understand that this can be achieved using a Watcher. While I’m able to implement the required functionality through Watcher, I’m facing challenges when it comes to integrating it with Microsoft Teams :sweat_smile:. I realize this might be veering into a different topic—my apologies for that.

Given that Kibana Alerting provides seamless integration with Teams, I thought of using it to fetch the last timestamp count instead.

Where I’m currently stuck with Teams integration:
I’m unsure about what values to provide for the host, port, and path fields in the payload. All I have is the webhook URL generated from the Teams channel.

"actions" : {
  "my_webhook" : {
    "transform" : { ... },
    "throttle_period" : "5m",
    "webhook" : {
      "method" : "POST",
      "host" : "mylisteningserver",
      "port" : 9200,
      "path": "/{{ctx.watch_id}}",
      "body" : "{{ctx.watch_id}}:{{ctx.payload.hits.total}}"
    }
  }
}

Could you please let me know if there’s an alternative approach to tackle this issue?

Thanks in advance!

4

Hello @mosaadshaikh1998

As a quick resolution , in action i will send an email to DL/myself. Using a powerautomate flow i will paste the data from the email to the TeamsChat.

For permanent fix you can check the documentation related to Teams Configuration :

Thanks!!