I'm trying to write a Watch that sends an email alert if the hard drive on the system I have metricbeat installed on starts getting full. I'm new to writing Watches and am struggling to decipher the error messages. At best guess I'm either missing a comma or a curly brace, or I've assumed I can use a field where I can't.
From elasticsearch.log
[2019-07-02T18:06:36,370][ERROR][o.e.x.w.i.s.ExecutableSearchInput] [ELK-test] failed to execute [search] input for watch [inlined], reason [[bool] malformed query, expected [END_OBJECT] but found [FIELD_NAME]]
From the Kibana Simulation Results (The curly brace belongs with the results but I lose formatting if I delete the line between it and the next line)
{
"watch_id": "_inlined_",
"node": "EaUsmy1eSwGLoX1AQzOwGQ",
"state": "executed",
"status": {
"state": {
"active": true,
"timestamp": "2019-07-03T01:06:36.369Z"
},
"last_checked": "2019-07-03T01:06:36.369Z",
"last_met_condition": "2019-07-03T01:06:36.369Z",
"actions": {
"send_email": {
"ack": {
"timestamp": "2019-07-03T01:06:36.369Z",
"state": "awaits_successful_execution"
}
}
},
"execution_state": "executed",
"version": -1
},
"trigger_event": {
"type": "manual",
"triggered_time": "2019-07-03T01:06:36.369Z",
"manual": {
"schedule": {
"scheduled_time": "2019-07-03T01:06:36.369Z"
}
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"metricbeat-*"
],
"rest_total_hits_as_int": true,
"body": {
"query": {
"bool": {
"filter": [
{
"range": {
"@timestamp": {
"gte": "now-{{ctx.metadata.window_period}}"
}
}
}
]
},
"match": {
"system.filesystem.device_name": "/dev/mapper/centos-root"
}
}
}
}
}
},
"condition": {
"compare": {
"system.filesystem.available": {
"gte": 10737418240
}
}
},
"metadata": {
"window_period": "30s",
"xpack": {
"type": "json"
}
},
"result": {
"execution_time": "2019-07-03T01:06:36.369Z",
"execution_duration": 0,
"input": {
"type": "search",
"status": "failure",
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "[bool] malformed query, expected [END_OBJECT] but found [FIELD_NAME]",
"line": 1,
"col": 74
}
],
"type": "parsing_exception",
"reason": "[bool] malformed query, expected [END_OBJECT] but found [FIELD_NAME]",
"line": 1,
"col": 74
},
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"metricbeat-*"
],
"rest_total_hits_as_int": true,
"body": {
"query": {
"bool": {
"filter": [
{
"range": {
"@timestamp": {
"gte": "now-30s"
}
}
}
]
},
"match": {
"system.filesystem.device_name": "/dev/mapper/centos-root"
}
}
}
}
}
},
"condition": {
"type": "always",
"status": "success",
"met": true
},
"actions": []
},
"messages": [
"failed to execute watch input"
]
}
And the watch itself (Which again loses formatting if I remove the space between the curly bracket and the rest of the code
{
"metadata": {
"window_period":"30s"
},
"trigger" : {
"schedule" : {
"daily" : { "at" : ["midnight", "14:00", "19:00"] }
}
},
"input" : {
"search" :{
"request":{
"indices": ["metricbeat-*"],
"body" : {
"query" :{
"bool" :{
"filter": [{
"range" : {
"@timestamp":{
"gte": "now-{{ctx.metadata.window_period}}"
}
}
}]
},
"match" : {
"system.filesystem.device_name": "/dev/mapper/centos-root"
}
}
}
}
}
},
"condition" : {
"compare" : {
"system.filesystem.available" : {
"gte" : 10737418240
}
}
},
"actions" : {
"send_email" : {
"email" : {
"to" : "ahmiura@ucdavis.edu",
"subject" : "ELK Watcher - Canadensis drive full",
"body" : "Canadensis has {{system.filesystem.available}} free"
}
}
}
}