# Watcher fails when an external webservice is unreachable
**URL:** https://discuss.elastic.co/t/watcher-fails-when-an-external-webservice-is-unreachable/141507
**Category:** Elasticsearch
**Tags:** elastic-stack-alerting
**Created:** [July 25, 2018, 7:26am UTC](https://discuss.elastic.co/t/watcher-fails-when-an-external-webservice-is-unreachable/141507 "2018-07-25T07:26:44Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@JeffreyH1989](https://discuss.elastic.co/u/JeffreyH1989)
#### Post date: [July 25, 2018, 7:26am UTC](https://discuss.elastic.co/t/watcher-fails-when-an-external-webservice-is-unreachable/141507/1 "2018-07-25T07:26:45Z")
Hello,
For simple monitoring of our webservices, I'm creating some watchers based on response codes.
However, when a service is unreachable (not something like response 404 not found, but an offline server) watcher execution fails.
```
"type": "connect_timeout_exception",
"reason": "Connect to url:80 [url/ip] failed: connect timed out",
"caused_by": {
"type": "socket_timeout_exception",
"reason": "connect timed out"
```
I tried setting the condition as follows:
"condition": {
"compare": {
"ctx.input.status": {
"eq": "failure"
}
}
Still the entire watcher fails and no actions were executed.
A simple example without any conditions fails aswell:
{
"trigger": {
"schedule": {
"interval": "5m"
}
},
"input": {
"http": {
"request": {
"scheme": "http",
"host": "webservice:80",
"port": 80,
"method": "get",
"params": {},
"headers": {}
}
}
}
}
Are there any options for catching these exceptions and log them accordingly?
Thank you in advanced
---
### Author:  [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [July 25, 2018, 7:39am UTC](https://discuss.elastic.co/t/watcher-fails-when-an-external-webservice-is-unreachable/141507/2 "2018-07-25T07:39:10Z")
you could wrap the http input within a `chain` input, which catches all exceptions, but then you need to check the payload yourself. Also the exception is not properly available.
From my watcher/elasticsearch perspective I would use a dedicated system for monitoring, which inserts data into elasticsearch. Watcher then only queries Elasticsearch. This has a few advantages. First, you are decoupling information collection and alerting, which is important when you add more alerts/endpoints. You also dont have to worry about watches being stuck when trying to connect to endpoint, which potentially prevent other watches from executing, as they are blocking a threadpool.
The Elastic Stack already allows you do to exactly this. You can use [heartbeat](https://www.elastic.co/guide/en/beats/heartbeat/current/configuration-heartbeat-options.html) for the heavy lifting of connecting to other services, managing timeouts and then have the result indexed into Elasticsearch. Heartbeat supports ICMP, TCP, HTTP checks, which should be sufficient in your use-case.
Hope this helps!
--Alex
---
### Author:  [@JeffreyH1989](https://discuss.elastic.co/u/JeffreyH1989)
#### Post date: [July 25, 2018, 8:25am UTC](https://discuss.elastic.co/t/watcher-fails-when-an-external-webservice-is-unreachable/141507/3 "2018-07-25T08:25:26Z")
Thank you for the help. I'll test with the chain input since it's just a simple health check. Further diagnostics will be done with heartbeats in the future.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 22, 2018, 8:25am UTC](https://discuss.elastic.co/t/watcher-fails-when-an-external-webservice-is-unreachable/141507/4 "2018-08-22T08:25:26Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.