Metricbeat watch

[as_neocase]

Hello,

I'm trying to implement a watch trigger to be aware when my disk used space is greater than 80 % .

In order to simulate i used the following querie but everytime the result.met is always false.

Did someone can help me to find a good way to do that ?

{
"trigger": {
"schedule": {
"interval": "10m"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"metr*"
],
"types": [],
"body": {
"size": 0,
"query": {
"bool": {
"must": {
"term": {
"metricset.name": "filesystem"
}
},
"filter": {
"range": {
"system.filesystem.used.pct": {
"gte": 0.8
}
}
}
}
}
}
}
}
},
"condition": {
"compare": {
"system.filesystem.used.pct": {
"gt": 0
}
}
},
"actions": {
"my-logging-action": {
"logging": {
"level": "error",
"text": "WARNING drive {{system.filesystem.device_name}} on {{beat.hostname}} less than 20% free space."
}
}
}
}

[spinscale (Alexander Reelsen)]

Hey,

please format your messages in markdown, so they are easier to read, especially long code snippets.

Second, you can use the watch history or the Execute Watch API to check how the execution went step by step. You can paste the output here.

There is also a long blog post, how to debug your watches and gain better insights into watch execution. You can read it right here https://www.elastic.co/blog/watching-the-watches-writing-debugging-and-testing-watches

Lastly, there is an examples repo that contains a sample watch for filesystem usage, where you can maybe copy and paste some code from.

Hope this helps!

--Alex

[manankalra]

Hey, I'm having trouble with setting alerts for my memory usage. I'm using Metricbeat which is stashing only the memory metrics to Elasticsearch.

This is the JSON for the watch that I've configured:

{
"trigger" : {
    "schedule" : {
    "interval": "10s"
    }
},
"input" : {
    "search" : {
    "request": {
        "search_type": "query_then_fetch",
        "indices" : [ "metricbeat-2017.07.06" ],
        "types": [],
        "body": {
            "query": {
                "match_all": {}
            }
        }
      }
    }
},
"condition": {
    "compare": {
        "ctx.payload.hits.hits._source.system.memory.actual.used.pct": {
            "gt": 0.5
        }
    }
},
"actions": {
    "logging": {
        "logging": {
            "level": "info",
                "text": "MEMORY USAGE ALERT!!! GREATER THAN 50%."
        }
    }
} 
}

I'm using match_all{} because all my metricsets are of memory type. It shows that my memory usage is around 0.85 but still no actions are triggered and the condition.met is always false. Need help asap!

[spinscale (Alexander Reelsen)]

Hey,

please open new threads instead of reviving old ones.

As already mentioned in this thread: If you are stuck, the first thing should always be to check either the last watch history entry or use the Execute Watch API to check what happens during every step of execution.

If you check the condition part of that output, you will see, that the condition path does not resolve to anything. As ctx.payload.hits.hits is an array, you have to specify the element you want to acccess.

That said, the approach to this watch is not the right one. Querying all documents and seeing which ones are fulfilled in the response does not make sense. You should use proper filters in your documents, like filtering by date, filtering by memory used, so that you also prevent the same documents triggering an alert over and over again.

Take a look at the already mentioned examples repo.

--Alex

[manankalra]

I'm having a little problem with understanding the example filesystem watch. Can you please correct my JSON? I need a little help here.

In short, filtering the documents by timestamp, reducing the result size to 1 and referring to the first element of the ctx.payload.hits.hits array will do the job, right?