Webhook URL issue, always appending the query ? even without parameters

Elastic Stack Kibana
1

We setup an ElasticSearch watcher through the Kibana - ElasticSearch - Watcher - Watches.

I added the JSON and our 'params' section is empty, but the generated URL always has a '?' appended even if we don't want any query parameters. If I try to remove the 'params' section completely it gets put back in automatically. Has anyone encountered this issue? I feel like I'm overlooking something very simple but cannot find anything in the documentation.

"webhook": {
"scheme": "https",
"host": "server.com",
"port": 443,
"method": "post",
"path": "/testpath",
"params": {},
"headers": {
"Content-type": "application/json"
},
"body": "{"message_type": "CRITICAL","monitoring_tool": "Elastic Watcher","entity_id": "{{ctx.id}}","entity_display_name": "{{ctx.watch_id}}","state_message": "{{ctx.watch_id}}","elastic_watcher_payload": {{#toJson}}ctx.payload{{/toJson}} }"
}

2

Hi @sgrady,

What if you simply omit the params property, as in the examples here?

3

Hi Chris, thanks for taking the time to respond. I did in fact try that and
found very odd behavior.

I went into the editor in the Kibana UI and deleted that line, then I click
'Save'. If I then go back into the editor and look, that empty params block
has appeared again.

I also tried deleting it completely and adding in the JSON with the params
section removed - it pastes in at 38 total lines, and if I go back into the
editor again to verify the code, it is 39 lines long and the empty params
block has been inserted. I don't understand what is going on.

Is there something else that needs to be changed?

4

would you mind sharing the whole watch by using the get watch API in the console? And also share the output of the execute watch API?

Also, which version is this on?

5

I had trouble finding a definitive version on the website, I think it is
v6.0.2. This is in an account I created just last week.

From the GET call, I get the following JSON that still shows an empty
'params' section.

{
  "found": true,
  "_id": "TEST2",
  "status": {
    "state": {
      "active": true,
      "timestamp": "2018-03-05T22:42:25.035Z"
    },
    "last_checked": "2018-03-05T23:32:03.750Z",
    "last_met_condition": "2018-03-05T23:32:03.750Z",
    "actions": {
      "victorops": {
        "ack": {
          "timestamp": "2018-03-05T22:42:25.035Z",
          "state": "awaits_successful_execution"
        },
        "last_execution": {
          "timestamp": "2018-03-05T23:32:03.750Z",
          "successful": false,
          "reason": "received [404] status code"
        }
      }
    },
    "execution_state": "executed",
    "version": 85
  },
  "watch": {
    "trigger": {
      "schedule": {
        "interval": "60s"
      }
    },
    "input": {
      "http": {
        "request": {
          "scheme": "http",
          "host": "
elastic:COO7Ch7qrAXnPVIwov08iWuI@2f867613f1cea4d95b58e5bdc5d49899.us-west-2.aws.found.io
",
          "port": 9200,
          "method": "get",
          "path": "/_cluster/health",
          "params": {},
          "headers": {}
        }
      }
    },
    "condition": {
      "always": {}
    },
    "actions": {
      "victorops": {
        "webhook": {
          "scheme": "https",
          "host": "stalert-victorops-com-1mnhtnnwmozq.runscope.net",
          "port": 443,
          "method": "post",
          "path":
"/integrations/generic/20131114/alert/83d27235-9641-4e78-bcc0-bfd572625621/testroutekey",
          "params": {},
          "headers": {
            "Content-type": "application/json"
          },
          "body": """{"message_type": "CRITICAL","monitoring_tool":
"Elastic Watcher","entity_id": "{{ctx.id}}","entity_display_name":
"{{ctx.watch_id}}","state_message":
"{{ctx.watch_id}}","elastic_watcher_payload":
{{#toJson}}ctx.payload{{/toJson}} }"""
        }
      }
    },
    "metadata": {
      "name": "Test 2",
      "xpack": {
        "type": "json"
      }
    }
  }
}
6

I also attempted to use the PUT endpoint to add a new watcher. In the new
JSON I don't have the 'params' section at all. The PUT command comes back
as success, and I got back and look at the code it uploaded and the
'params' section has been inserted.

I don't understand why the '?' is appended to the URL or how to get rid of
it at this point.

7

Are there any other working webhook examples available?

I'm not finding a way around this params/'?' issue.

8

can you share the exact JSON you used to put the watch? also which elasticsearch version is this you are running this against?

Thank you!

9

I have the same issue here is a watcher with service api and routing key removed:

{
  "trigger": {
    "schedule": {
      "interval": "60s"
    }
  },
  "input": {
    "http": {
      "request": {
        "scheme": "http",
        "host": "localhost",
        "port": 9200,
        "method": "get",
        "path": "/_cluster/health",
        "params": {},
        "headers": {}
      }
    }
  },
  "condition": {
    "always": {}
  },
  "actions": {
    "victorops": {
      "webhook": {
        "scheme": "https",
        "host": "alert.victorops.com",
        "port": 443,
        "method": "post",
        "path": "/integrations/generic/20131114/alert/serviceapi/routingkey",
        "params": {},
        "headers": {
          "Content-type": "application/json"
        },
        "body": "{\"message_type\": \"CRITICAL\",\"monitoring_tool\": \"Elastic Watcher\",\"entity_id\": \"{{ctx.id}}\",\"entity_display_name\": \"{{ctx.watch_id}}\",\"state_message\": \"{{ctx.watch_id}}\",\"elastic_watcher_payload\": {{#toJson}}ctx.payload{{/toJson}} }"
      }
    }
  }
}
10

I am having a similar issue which Sean described above. I didn't use a PUT to create the watch, instead I built the entire watch using Kibana. I however seemed to have narrowed the issue to some sort of haproxy issue.

The alert seems to be directed to the entirely wrong url for some reason, thus yielding a 404 error. I should also note that when replicating the configuration as a curl it works just fine when posting to a victorops endpoint.

11

I found the issue regarding the question mark at the end of the path, even though there are no parameters. It's a bug. I'll fix it.

12

Great, thanks for checking that out.

There still seems to be some issue at play here from some further testing I performed. It looks on my side like the url is still getting routed incorrectly - almost like the host and is getting chopped off or something. We should be hitting “alert.server.com” for example to get routed correctly, and I can run a curl that works, but the one from kibana shows a 404 in the UI.

Is there anything else that happens to the url info before posting to the webhook? Is it possible to see logs of exactly what kibana was trying to hit?

1 Like
13

you can use the execute watch API and paste the output (in a formatted way, please), and I'm happy to take a look!

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.