Watcher: Generating URLs to link back to Watch History

[AndyBCDL (Andy Brown)]

Hi folks, we've just upgraded to Elastic 5 over Dev, UAT and Live. I'm just looking at Watcher for the first time.

I've configured a simple query to look for HTTP 503 errors in our HAproxy access logs and report to our slack channel if there are more than 25 errors over a 5 minute period. Here is the JSON

{
"trigger": {
"schedule": {
"interval": "5m"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"logstash-haproxy_access-*"
],
"types": ,
"body": {
"query": {
"bool": {
"must": {
"match": {
"http_status_code": 503
}
},
"filter": {
"range": {
"@timestamp": {
"from": "{{ctx.trigger.scheduled_time}}||-5m",
"to": "{{ctx.trigger.triggered_time}}"
}
}
}
}
}
}
}
}
},
"condition": {
"compare": {
"ctx.payload.hits.total": {
"gte": 25
}
}
},
"actions": {
"notify-slack": {
"throttle_period_in_millis": 300000,
"slack": {
"account": "REMOVEDCHANNELNAME",
"message": {
"from": "kibana-dev",
"to": [
"#kibana-dev"
],
"attachments": [
{
"color": "danger",
"title": "HAPROXY 503 ERRORS",
"text": "Encountered {{ctx.payload.hits.total}} HTTP 503 errors in the last 5 minutes"
}
]
}
}
}
}
}

And what you see in slack

So far, so good.

However, what i'd like to do is to add a URL to the slack message which links back to the Watch History or a Watcher index so someone can clickthrough and investigate the messages, then mark the slack message as seen / green tick to indicate its been investigated.

I am not sure where to start. Any ideas?

[spinscale (Alexander Reelsen)]

Hey,

would linking to the watcher UI work for you? sth like

http://localhost:5601/app/kibana#/management/elasticsearch/watcher/watches/watch/YOUR_WATCH_ID/status?_g=()

will show you directly the latest history entries of that particular watch.

--Alex

[AndyBCDL (Andy Brown)]

Its a good starter for 10 for sure, but as time goes one we'd probably want to be able to specifically grab the information that caused the alert.

I had a few ideas over the weekend but not sure how feasible they are;

1. Putting the alert into an index and then linking to that index
2. Creating a discover URL with the same timescale and query value used in watcher