Hi,
I have watches pointing at our main (non marvel) cluster and logging results was watch_records in an index we created for this purpose. Eventually we're going to hook this up to a webhook, but for now just want a log. Here's the watch definition:
PUT _watcher/watch/cluster_health_watch_clusterisredtowatcherlog
{
"trigger" : {
"schedule" : { "interval" : "60s" }
},
"input" : {
"http" : {
"request" : {
"host" : "localhost",
"port" : 9200,
"path" : "/_cluster/health"
}
}
},
"throttle_period": "30m",
"condition" : {
"compare" : {
"ctx.payload.status" : { "eq" : "red" }
}
},
"actions" : {
"index_payload" : {
"index" : {
"index" : "watch_record_log",
"doc_type" : "watch_record"
}
}
}
}
I'd like to do the same thing for high memory utilization. Using https://www.elastic.co/guide/en/watcher/2.3/watching-marvel-data.html as a guide, I tried:
PUT _watcher/watch/mem_watch
{
"trigger": {
"schedule": {
"interval": "1m"
}
},
"input": {
"search": {
"request": {
"indices": [
".marvel*"
],
"types" : [
"node_stats"
],
"body": {
"size" : 0,
"query": {
"bool": {
"filter": {
"range": {
"timestamp": {
"gte": "now-2m",
"lte": "now"
}
}
}
}
},
"aggs": {
"minutes": {
"date_histogram": {
"field": "timestamp",
"interval": "minute"
},
"aggs": {
"nodes": {
"terms": {
"field": "source_node.name",
"size": 10,
"order": {
"memory": "desc"
}
},
"aggs": {
"memory": {
"avg": {
"field": "node_stats.jvm.mem.heap_used_percent"
}
}
}
}
}
}
}
}
}
}
},
"throttle_period": "30m",
"condition": {
"script": "if (ctx.payload.aggregations.minutes.buckets.size() == 0) return false; def latest = ctx.payload.aggregations.minutes.buckets[-1]; def node = latest.nodes.buckets[0]; return node && node.memory && node.memory.value >= 75;"
},
"actions" : {
"index_payload" : {
"index" : {
"index" : "watch_record_log",
"doc_type" : "watch_record"
}
}
}
}
But, I'm getting:
"messages": [
"failed to execute watch input"
],
"result": {
"execution_time": "2016-08-02T19:25:17.111Z",
"execution_duration": 157,
"input": {
"type": "search",
"status": "failure",
"reason": "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[nZOwlZoDSMW--YR_ZlqePQ][.marvel-2016.06.01][0]: RemoteTransportException[[marvelnode][inet[/xx.x.x.xx:xxxx]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[.marvel-2016.06.01][0]: from[-1],size[0]: Parse Failure [Failed to parse source [{"size":0,"query":{"bool":{"filter":{"range":{"timestamp":{"gte":"now-2m","lte":"now"}}}}},"aggs":{"minutes":{"date_histogram":{"field":"timestamp","interval":"minute"},"aggs":{"nodes":{"terms":{"field":"source_node.name","size":10,"order":{"memory":"desc"}},"aggs":{"memory":{"avg":{"field":"node_stats.jvm.mem.heap_used_percent"}}}}}}}}]]]; nested: QueryParsingException[[.marvel-2016.06.01] [bool] query does not support [filter]]; }
GET /.marvel*/node_stats/_search returns 6717451 results, and I see the heap_used_percent values.
If I remove the bool parameter, the error says "no query registered for [filter]".
Thanks,
Casie