Watcher: Failed to send sample slack message (reopening)

Elastic Stack Elasticsearch
1

[context]:

Hi there,

I m having the exact same issue that happened here :

I already verified that I can push a message to slack through the webhook I have.

I have added this in my user settings overrides:

xpack.notification.slack:
  account:
    monitoring:
      url: https://hooks.slack.com/services/XXXXXX/XXXXXXX/XXXxxXXXXXXxxxXx

I also ran the watcher's execution command that did return this:

{"_id":"XXXXXXX-f8f71ec840f1-2019-01-29T10:45:50.201Z","watch_record":{"watch_id":"b9380bdd-878e-4832-9f5c-54642a9e66f2","node":"8tnzogvWQ0GmmzDZh6uy-A","state":"executed","user":"elastic","status":{"state":{"active":true,"timestamp":"2019-01-29T10:45:40.822Z"},"last_checked":"2019-01-29T10:45:50.201Z","last_met_condition":"2019-01-29T10:45:50.201Z","actions":{"email_1":{"ack":{"timestamp":"2019-01-29T10:45:50.201Z","state":"ackable"},"last_execution":{"timestamp":"2019-01-29T10:45:50.201Z","successful":true},"last_successful_execution":{"timestamp":"2019-01-29T10:45:50.201Z","successful":true}},"slack_1":{"ack":{"timestamp":"2019-01-29T10:45:40.822Z","state":"awaits_successful_execution"},"last_execution":{"timestamp":"2019-01-29T10:45:50.201Z","successful":false,"reason":""}}},"execution_state":"executed","version":1170},"trigger_event":{"type":"manual","triggered_time":"2019-01-29T10:45:50.201Z","manual":{"schedule":{"scheduled_time":"2019-01-29T10:45:50.201Z"}}},"input":{"search":{"request":{"search_type":"query_then_fetch","indices":["backend-dev_warning*"],"types":[],"body":{"size":0,"query":{"bool":{"filter":{"range":{"utctime":{"gte":"{{ctx.trigger.scheduled_time}}||-5m","lte":"{{ctx.trigger.scheduled_time}}","format":"strict_date_optional_time||epoch_millis"}}}}}}}}},"condition":{"script":{"source":"if (ctx.payload.hits.total > params.threshold) { return true; } return false;","lang":"painless","params":{"threshold":2}}},"metadata":{"name":"Dev Warning threshold","watcherui":{"trigger_interval_unit":"m","agg_type":"count","time_field":"utctime","trigger_interval_size":1,"term_size":5,"time_window_unit":"m","threshold_comparator":">","term_field":null,"index":["backend-dev_warning*"],"time_window_size":5,"threshold":2,"agg_field":null},"xpack":{"type":"threshold"}},"result":{"execution_time":"2019-01-29T10:45:50.201Z","execution_duration":519,"input":{"type":"search","status":"success","payload":{"_shards":{"total":5,"failed":0,"successful":5,"skipped":0},"hits":{"hits":[],"total":0,"max_score":0.0},"took":2,"timed_out":false},"search":{"request":{"search_type":"query_then_fetch","indices":["backend-dev_warning*"],"types":[],"body":{"size":0,"query":{"bool":{"filter":{"range":{"utctime":{"gte":"2019-01-29T10:45:50.201Z||-5m","lte":"2019-01-29T10:45:50.201Z","format":"strict_date_optional_time||epoch_millis"}}}}}}}}},"condition":{"type":"always","status":"success","met":true},"transform":{"type":"script","status":"success","payload":{"result":0}},"actions":[{"id":"slack_1","type":"slack","status":"failure","slack":{"account":"monitoring","sent_messages":[{"status":"failure","request":{"host":"hooks.slack.com","port":-1,"scheme":"https","method":"post","headers":{"Content-Type":"application/json; charset=UTF-8"},"body":"{\"channel\":\"https://hooks.slack.com/services/XXXXXX/XXXXXX/XXXXXXXXXXXXXXXX\",\"username\":\"68fdsgdf8-878e-4832-9f5c-654684sdf\",\"text\":\"Watch [Dev Warning threshold] has exceeded the threshold\"}"},"response":{"status":404,"headers":{"date":["Tue, 29 Jan 2019 10:45:50 GMT"],"server":["Apache"],"transfer-encoding":["chunked"],"vary":["Accept-Encoding"],"x-frame-options":["SAMEORIGIN"],"x-via":["haproxy-www-4yop"],"strict-transport-security":["max-age=31536000; includeSubDomains; preload"],"via":["1.1 9645d960e68082d4a7cc7b8be082e4ae.cloudfront.net (CloudFront)"],"access-control-allow-origin":["*"],"referrer-policy":["no-referrer"],"connection":["keep-alive"],"content-type":["text/html"],"x-cache":["Error from cloudfront"],"x-amz-cf-id":["VV60gqKyQubPw7ZReFSSMTJB7-rCyfJ34vhizlc3o1HCPu7Z_MwE8A=="]},"body":"channel_not_found"},"to":"https://hooks.slack.com/services/XXXXXX/XXXXXX/XXXXXXXXXXXXXXXX","message":{"from":"68fdsgdf8-878e-4832-9f5c-654684sdf","text":"Watch [Dev Warning threshold] has exceeded the threshold"}}]}}]},"messages":[]}}

I analysed the return of the watcher execution command : "body": "channel_not_found"

Thank you in advance for your help.

2

Hi there,

I just followed up in your current support case but wanted to close the loop here as well. The key to the failure is included in the watcher response body:

"body":"channel_not_found"

Within the request body of the response payload it's indicating that the "message.to" option is currently set to your integrations webhook URL :

"body": "{\"channel\":\"https://hooks.slack.com/services/XXXXXX/XXXXXX/XXXXXXXXXXXXXXXX\",\"username\":\"68fdsgdf8-878e-4832-9f5c-654684sdf\",\"text\":\"Watch [Dev Warning threshold] has exceeded the threshold\"}"

Once the Watcher action is updated to point to the desired channel or user accounts within your Slack org, you'll be good to go.

https://www.elastic.co/guide/en/x-pack/current/actions-slack.html#slack-action-attributes

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.