Watcher issue (for real)

Elastic Stack Elasticsearch
1

The examples on elastic.co seem to work unmodified in my environment, I am however having an issue:

This works and sends email:
0}[root@test75 ~]# curl -XPUT 'http://test45:9200/_watcher/watch/cluster_health_watch7' -d '{

"trigger" : {
"schedule" : { "cron" : "0 0/1 * * * ?" }
},
"input" : {
"search" : {
"request" : {
"indices" : [
"filebeat*"
"body" : {
],
"body" : {
"filtered": {
"query" : {
"filtered": {
},
"query": {
"match": { "response": 404 }
},
"filter": {
"range": {
"@timestamp" : {
"from": "{{ctx.trigger.scheduled_time}}||-5m",
"to": "{{ctx.trigger.triggered_time}}"
}
}
}
}
}
}
}
}
},
"actions" : {
"email_admin" : {
"email" : {
"to" : "kartik.unix@gmail.com",
"subject" : "404 recently encountered"
}
}
}
}'
{"_id":"cluster_health_watch7","_version":1,"created":true}[root@test75 ~]#

This does not:

PUT _watcher/watch/my-watch8
{
"trigger" : {
"schedule" : { "cron" : "0 0/1 * * * ?" }
},
"input" : {
"search" : {
"request" : {
"indices" : [
"filebeat*"
],
"body" : {
"query" : {
"filtered": {
"query": {
"match": { "response": 404 }
},
"filter": {
"range": {
"@timestamp" : {
"from": "{{ctx.trigger.scheduled_time}}||-5m",
"to": "{{ctx.trigger.triggered_time}}"
}
}
}
}
}
}
}
}
},
"condition" : {
"script" : "ctx.payload.hits.total > 1"
},
"actions" : {
"email_admin" : {
"email" : {
"to" : "kartik.unix@gmail.com",
"subject" : "404 recently encountered"
}
}
}
}'

2

What doesn't work?
Do you get an error?
What is the error?

Please provide more information.

3

Well, first allow me to thank you for your interest.
To generate an email I browse to a non existent page, in the first instance I get an email alert, not so in the second instance. I think it has something to do with the: ctx.payload.hits.total > 1" which I am introducing only to be alerted for "real" alerts.......

4

Hi,

You should check if the second time your browse to the non existing page a corresponding document has been created in your index.

If so, you can try to run the watch search request to check that the document is correctly found.

If so, you can check in the .watch-history index if the watch has been triggered and what was the result of the execution.

5

Since this code is verbatim from elastic.co, I'd much rather someone tell me if there's something I am doing wrong, the only thing that stands out is that I am using filebeat instead of logstash, but is not "filebeat" a step up...........

any answers will be most appreciated.

6

If I remove:
"condition" : {
"script" : "ctx.payload.hits.total > 1"
},
then I constantly get alerted.

This is what I see in Kibana, here I am sending from filebeat to logstash and from logstash to ES:
http://pastebin.com/20LZMgku

7

figured this one out:
works with logstash output to ES , I'm using logstash* as index