Harper_S1
August 1, 2025, 7:50pm
1
Hi , I am trying to send health check up through watcher to splunk but getting ssl exception while executing watcher
{
"trigger": {
"schedule": {
"cron": "0 0 9 ? * *"
}
},
"input": {
"http": {
"request": {
"scheme": "https",
"host": "someElastichost",
"port": 9200,
"method": "get",
"path": "/_cluster/health",
"params": {},
"headers": {},
"auth": {
"basic": {
"username": "elastic",
"password": "somepassword"
}
},
"connection_timeout_in_millis": 30000
}
}
},
"condition": {
"always": {}
},
"actions": {
"send_to_splunk": {
"webhook": {
"scheme": "https",
"host": "splunkHOST",
"port": 0909,
"method": "post",
"path": "/_cluster/health",
"params": {},
"headers": {
"Authorization": "somepassword",
"Content-Type": "application/json"
},
"body": """{"message_type": "CRITICAL","monitoring_tool": "Elastic Watcher","entity_id": "{{ctx.id}}","entity_display_name": "{{ctx.watch_id}}","state_message": "{{ctx.watch_id}}","elastic_watcher_payload": {{#toJson}}ctx.payload{{/toJson}} }"""
}
}
}
}
Getting the below error
"actions": [
{
"id": "send_to_splunk",
"type": "webhook",
"status": "failure",
"error": {
"root_cause": [
{
"type": "s_s_l_exception",
"reason": "Unsupported or unrecognized SSL message"
}
],
"type": "s_s_l_exception",
"reason": "Unsupported or unrecognized SSL message"
}
}
]
},
Harper_S1
August 1, 2025, 9:35pm
2
I have figured this out, instead of https, http is working but now getting another error.
"body": "{\"text\":\"Invalid data format\",\"code\":6,\"invalid-event-number\":0}"
thanks!
Harper_S1
August 1, 2025, 10:42pm
3
{\"unassigned_primary_shards\":0,\"number_of_pending_tasks\":0,\"cluster_name\":\"poc_es\",\"active_shards\":52,\"active_primary_shards\":52,\"unassigned_shards\":0,\"delayed_unassigned_shards\":0,\"timed_out\":false,\"relocating_shards\":0,\"_headers\":{\"x-elastic-product\":[\"Elasticsearch\"],\"content-type\":[\"application/json\"]},\"initializing_shards\":0,\"task_max_waiting_in_queue_millis\":0,\"number_of_data_nodes\":1,\"number_of_in_flight_fetch\":0,\"active_shards_percent_as_number\":100.0,\"_status_code\":200,\"status\":\"green\",\"number_of_nodes\":2}"
anyone knows about how we can structured the above payload to json.
I tried {{#toJson}}ctx.payload{{#toJson}}, but same issue.
Hello @Harper_S1
As per documentation we can try using : response_content_type
"input": {
"http": {
"request": {
"scheme": "https",
"host": "my-elk.com",
"port": 443,
"method": "get",
"path": "/_cluster/health",
"params": {},
"headers": {
"Authorization": "ApiKey mykey"
},
"connection_timeout_in_millis": 30000
},
"response_content_type": "json"
}
}
This is valid
"elastic_watcher_payload": {{#toJson}}ctx.payload{{/toJson}}
Thanks!!
Harper_S1
August 5, 2025, 5:08pm
5
Thanks, I tried above suggested, but seems additional backslash is not allowing to send it as json.
"body": "{\"unassigned_primary_shards\":0,\"number_of_pending_tasks\":0,\"cluster_name\":\"poc_es_8\",\"active_shards\":55,\"active_primary_shards\":55,\"unassigned_shards\":0,\"delayed_unassigned_shards\":0,\"timed_out\":false,\"relocating_shards\":0,\"_headers\":{\"x-elastic-product\":[\"Elasticsearch\"],\"content-type\":[\"application/json\"]},\"initializing_shards\":0,\"task_max_waiting_in_queue_millis\":0,\"number_of_data_nodes\":1,\"number_of_in_flight_fetch\":0,\"active_shards_percent_as_number\":100.0,\"_status_code\":200,\"status\":\"green\",\"number_of_nodes\":2}"
},
and , giving this error
"body": "{\"text\":\"No data\",\"code\":5}"
Thanks @Harper_S1 for the update.
As part of the Splunk official documentation
undefined
It is using path :
"path": "/integrations/generic/20131114/alert/$service_api_key/$routing_key",
In your actions path :
"path": "/_cluster/health"
I am not sure is it because of this configuration, could you please check?
As a workaround, we can try sending the data to Elasticsearch as part of the action to elastic index, for example, test_splunk to observe how the data is stored. This might help us understand why the same information isn't being successfully sent to Splunk.
Thanks!!
Harper_S1
August 6, 2025, 4:23pm
7
sorry , first code had some typos. Below one is the latest.
{
"trigger": {
"schedule": {
"cron": "0 0 9 ? * *"
}
},
"input": {
"http": {
"request": {
"scheme": "https",
"host": "elasticHOST",
"port": 9200,
"method": "get",
"path": "/_cluster/health",
"params": {},
"headers": {},
"auth": {
"basic": {
"username": "elastic",
"password": "somepassword"
}
},
"connection_timeout_in_millis": 30000
}
}
},
"condition": {
"always": {}
},
"actions": {
"send_to_splunk": {
"webhook": {
"scheme": "http",
"host": "splunkHOST",
"port": somePORT,
"method": "post",
"path": "/services/collector/event",
"params": {},
"headers": {
"Authorization": "someToken"
},
"body": "{{#toJson}}ctx.payload.requestBody{{/toJson}}"
}
}
},
"transform": {
"script": {
"source": """ctx.payload.requestBody = ctx.payload.replaceAll('\\\\', '');""",
"lang": "painless"
}
}
}
I am trying to use transform as well but no luck!
Harper_S1
August 7, 2025, 8:18pm
8
By using event field under the body of webhook, helped to send data.
Here is the working request.
{
"trigger": {
"schedule": {
"cron": "0 0/1 * * * ?"
}
},
"input": {
"http": {
"request": {
"scheme": "https",
"host": "ELASTIC.COM",
"port": 9200,
"method": "get",
"path": "/_cluster/health",
"params": {},
"headers": {},
"auth": {
"basic": {
"username": "elastic",
"password": "SOMEPASSWORD"
}
},
"connection_timeout_in_millis": 3000
}
}
},
"condition": {
"always": {}
},
"actions": {
"send_to_splunk": {
"webhook": {
"scheme": "http",
"host": "SPLUNK.COM",
"port": 7007,
"method": "post",
"path": "SOMEPATH",
"params": {},
"headers": {
"Authorization": "SOMPASSWORD",
"Content-Type": "application/json"
},
"body": """{"event": "{{ctx.payload}}"}"""
}
}
}
}