Create alerts in packetbeat monitoring

Elastic Stack Beats
,
1

I am monitoring packetbeat using xpack. Monitoring beats,if my packetbeat goe's down i should get notified through mail,how could i create watcher alert in this scenario.

Thanks
Ramya

2

Hi @RamyaGowda,

You first need to find a condition upon which you want to alert. For example, if you have received less than n documents within the last m minutes, then an email should be sent.

You will find all technical details, including information on how to send email, in the docs about Watcher: https://www.elastic.co/guide/en/elastic-stack-overview/current/watcher-getting-started.html. Hope this helps!

Florian

3

Hi @fkelbert

Assume that i have 10 packetbeats running and 2 of them goes down at that time i would like send an alert/notification through mail saying that packetbeat is down in this host(IP address)

so that when i get alerts i can go and check my agents why they are stopped

bellow is the sample data what i have in .monitoring-beats-* index

"_index" : ".monitoring-beats-6-2019.02.12",
"_type" : "doc",
"_id" : "GKAB32gBPlvhczozsqzy",
"_score" : 1.0,
"_source" : {
  "cluster_uuid" : "4u5N-s_6TVKXmCRE6w8ecw",
  "timestamp" : "2019-02-12T00:00:57.070Z",
  "interval_ms" : 60000,
  "type" : "beats_state",
  "source_node" : {
    "uuid" : "6bqfOxWeSS6NHdwhLUH54Q",
    "host" : "x.x.x.x",
    "transport_address" : "localhost:9300",
    "ip" : "localhost",
    "name" : "6bqfOxW",
    "timestamp" : "2019-02-12T00:00:57.070Z"
  },
  "beats_state" : {
    "timestamp" : "2019-02-12T00:00:57.067Z",
    "beat" : {
      "host" : "host1",
      "uuid" : "ea9ab830-4418-4dde-9738-ab84510687fc",
      "type" : "packetbeat",
      "version" : "6.5.4",
      "name" : "abc"
    },
    "state" : {
      "beat" : {
        "name" : "host1"
      },
      "host" : {
        "architecture" : "x86_64",
        "os" : {
          "platform" : "rhel",
          "version" : "7.4 (Maipo)",
          "family" : "",
          "codename" : "Maipo"
        },
        "id" : "dcc182a64f544eeebedf7d8df588452b",
        "containerized" : "containerized",
        "name" : "host1"
      },
      "management" : {
        "enabled" : false
      },
      "queue" : {
        "name" : "mem"
      },
      "output" : {
        "name" : "elasticsearch"
      },
      "service" : {
        "version" : "6.5.4",
        "name" : "packetbeat",
        "id" : "0d4e7da2-f933-4ae9-9b92-56a4c613abc3"
      }
    }
  }
}
4

You could, for example, run a terms aggregation over the field source_node.name or source_node.host and then compare the returned list against the list of hosts that you are expecting, e.g., using a Painless script within the Watcher condition.

All information about Watchers and their parts are detailed in the Watcher API. If you have an Elasticsearch subscription (which you seem to have), you might also want to consult Elastic Support with any concrete Watchers that you might have trouble with.

5

Ok if i write Aggregation query like bellow

{
"query": {
"bool": {
"must": [
{
"match_all": {}
}
],
"filter": {
"range": {
"timestamp": {
"gte": "now-15m",
"lte": "now"
}
}
}
}
},
"aggs": {
"AggregatedData": {
"terms": {
"field": "beats_stats.beat.name"
}
}
}
}

outcome is:

{
"aggregations" : {
"AggregatedData" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "beat1",
"doc_count" : 90
},
{
"key" : "beat2",
"doc_count" : 6
}
]
}
}
}

if i change my filter query to check for last 1 min data

{
"query": {
"bool": {
"must": [
{
"match_all": {}
}
],
"filter": {
"range": {
"timestamp": {
"gte": "now-1m",
"lte": "now"
}
}
}
}
},
"aggs": {
"AggregatedData": {
"terms": {
"field": "beats_stats.beat.name"
}
}
}
}

i am not gwtting host1 in result because beat1 is down from last few minutes:

{
"aggregations" : {
"AggregatedData" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "beat2",
"doc_count" : 6
}
]
}
}
}

how do i compare this two queries result , and moreover how to configure it in watcher.
Sorry i don't have much idea about creating watcher seeking for help.

Thanks
Ramya

6

Watcher input: For the Watcher input, you would use a chain input. A chain input allows you to execute multiple inputs of type search. These searches would be the two aggregations that you detailed above.

Watcher condition: Within the condition, you would use a script condition. Within a script, you can use the Painless scripting language to compare the two result sets from the above searches.

7

Hi @fklbert,

Thank you for your help!

I have tried Configuring watcher ,able to use chain input properly.
Not understanding how to compare two aggregations result using Painless scrpting lauguage.:frowning:

Configured watcher like bellow:

{
"trigger": {
"schedule": {
"interval": "1m"
}
},
"input": {
"chain": {
"inputs": [
{
"first": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
".monitoring-beats-6-2019.02.14"
],
"types": ,
"body": {
"query": {
"bool": {
"must": [
{
"match_all": {}
}
]
}
},
"aggs": {
"AggregatedData": {
"terms": {
"field": "beats_stats.beat.name"
}
}
},
"_source": [
"message"
]
}
}
}
}
},
{
"second": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
".monitoring-beats-6-2019.02.14"
],
"types": ,
"body": {
"query": {
"bool": {
"must": [
{
"match_all": {}
},
{
"filter": {
"range": {
"timestamp": {
"gte": "now-5m",
"lte": "now"
}
}
}
}
]
}
},
"aggs": {
"AggregatedData": {
"terms": {
"field": "beats_stats.beat.name"
}
}
},
"_source": [
"message"
]
}
}
}
}
}
]
}
},
"condition": {
"script": {
"source": "return result",
"lang": "painless"
}
},
"actions": {
"email_users": {
"email": {
"profile": "standard",
"priority": "high",
"to": [
"ramya.cp@gmail.com"
],
"subject": "PACKETBEAT ALERT: in this host xxxx packetbeat is not running since x minute/hour!",
"body": {
}
}
}
}
}

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.