Notify Slack action in watcher fails if message attachment contains actions

alerting

(Victor) #1

Elastic cluster 6.3.2 is deployed to Elastic cloud.

While trying to animate the Slack notifications sent by our watchers I have updated the message to include a link button as per Slack documentation . As a result, the code looks like that

...
    "actions": {
        "notify-slack": {
            "throttle_period_in_millis": 600000,
            "slack": {
                "account": "monitoring",
                "message": {
                    "from": "watcher",
                    "to": [
                        "#slack-channel"
                    ],
                    "text": "Something bad has happened",
                    "attachments": [
                        {
                            "color": "danger",
                            "fallback": "During last 5 mins, something bad has happened - Server has reported {{ctx.payload.hits.total}} times for {{ctx.payload.aggregations.address_pairs_count.value}} route(s) that it is not working - Check service dashboard https://dashboard.server.com/?service=bla-bla-bla",
                            "pretext": "During last 5 mins, something bad has happened",
                            "title": "It is not working",
                            "text": "Server has reported *{{ctx.payload.hits.total}} times* for *{{ctx.payload.aggregations.address_pairs_count.value}} route(s)* that it is not working",
                            "mrkdwn_in": [
                                "text"
                            ],
                            "actions": [
                                {
                                    "type": "button",
                                    "text": "Service dashboard",
                                    "url": "https://dashboard.server.com/?service=bla-bla-bla"
                                }
                            ]
                        }
                    ]
                }
            }
        }
    }
}

Even though the code editor parsed the code fine and the watcher status is shown as "Ok", every attempt to open that watcher fails with the response HTTP 400 (Bad Request)

{"statusCode":400,"error":"Bad Request","message":"[parse_exception] failed to parse [slack] action [my-watcher/notify-slack]. failed to parse [message] field"}

Additional experiments have shown that the problem appears as soon as I add the "actions" array into message "attachments". My guess that the watcher is mixing up its own actions and the Slack message actions.

Did anybody see that before or knows a workaround? Any ideas are very much welcome!


(Alexander Reelsen) #2

Hey,

unfortunately we have a strict parsing mechanism currently with slack messages, that is a bit behind the features the slack actions allow you to do. What this means is, that Slack at some point added a type: button field to their API, which we have not added to our watch parsing mechanism. This is the reason for the failure. If you omit the type field, everything should work - but of course you are missing the button in the slack message.

There is an issue for this already, see https://github.com/elastic/elasticsearch/issues/31032

--Alex


(Victor) #3

Hi Alex,

Thank you for pointing out the GitHub issue! It did not occur to me to check the elasticsearch project.

I'll avoid using buttons for now until that issue is fixed.

Victor


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.