Watcher disk usage alerting

[ss02072805 (SABARI DOSS GOVINDARAJ)]

This is for Metricbeat filesystem used, whenever threshold level goes above 40 % it should trigger alerts ,

**Execution failing:**
"error": {
"root_cause": [
{
"type": "general_script_exception",
"reason": "Error running com.github.mustachejava.codes.DefaultMustache@6eb548c2"
}
],
"type": "general_script_exception",
"reason": "Error running com.github.mustachejava.codes.DefaultMustache@6eb548c2",
"caused_by": {
"type": "mustache_exception",
"reason": "Failed to get value for ctx.payload.hits.hits.0._source.host.name @[query-template:1]",
"caused_by": {
"type": "mustache_exception",
"reason": "0 @[query-template:1]",
"caused_by": {
"type": "index_out_of_bounds_exception",
"reason": "0"

___________________________________________________________________________________________________Watcher disk usage query

Blockquote

{
"trigger": {
"schedule": {
"interval": "2m"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"metricbeat-*"
],
"rest_total_hits_as_int": true,
"body": {
"size": 0,
"query": {
"bool": {
"filter": [
{
"range": {
"@timestamp": {
"from": "now-15m"
}
}
},
{
"range": {
"system.filesystem.used.pct": {
"from": 0.45
}
}
}
],
"must": [
{
"match_phrase": {
"system.filesystem.mount_point": "/MDHUATAP001"
}
}
]
}
},
"aggs": {
"by_host": {
"terms": {
"field": "host.hostname",
"size": "100"
}
},
"by_disk": {
"terms": {
"field": "system.filesystem.mount_point",
"size": "100"
}
}
}
}
}
}
},
"condition": {
"compare": {
"ctx.payload.hits.total": {
"gt": 0
}
}
},
"actions": {
"send_email": {
"throttle_period_in_millis": 9000,
"email": {
"profile": "standard",
"to": [
"sabaridoss.govindaraj@satr.com",
"sabaridoss.govindaraj@satr.com"
],
"subject": "[SSM][NON PROD][{{ctx.payload.hits.hits.0._source.host.name}}] Application restarted",
"body": {
"testing"
}
}
}
}
}

Blockquote

[spinscale (Alexander Reelsen)]

Hey,

please use some markdown formatting for your JSON, as this is really hard to read.

Can you also share the output of the execute watch API or the watcher history for a single execution that failed? That would be great!

Current guess: Is it possible that the first document returned does not have the ctx.payload.hits.hits.0._source.host.name (so the host.name field) in its source?

--Alex