# SSLHandshakeException while using webhook action in watcher **URL:** \ **Category:** Elasticsearch\ **Created:** [July 28, 2017, 6:18am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906 "2017-07-28T06:18:26Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![biswars](https://avatars.discourse-cdn.com/v4/letter/b/d9b06d/32.png) [@biswars](https://discuss.elastic.co/u/biswars)\ **Post date:** [July 28, 2017, 6:18am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/1 "2017-07-28T06:18:26Z")
Version: 5.4.2 I do have the CA cert used in the target host in the webhook action url and it is tested using curl --cacert option and works well. Followed the followings > **[Watcher Settings in Elasticsearch | Elasticsearch Reference \[5.5\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/notification-settings.html#ssl-notification-settings)** > [@Webhook https , ignoring verification of certificate](https://discuss.elastic.co/t/webhook-https-ignoring-verification-of-certificate/25874/6): > > Hey, using the watcher.http.ssl.keystore.{path,password,key\_password,algorithm} settings, one can also use a dedicated keystore for the http input/webhook action, which is also used as truststore, unless you are using shield, which uses it's own. --Alex However, I still get "SSLHandshakeException[Received fatal alert: handshake\_failure]" in the response As per the logs (we can see below) the error seems to be in the permission of the trust store file: Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "/etc/pki/java/cacerts" "read") But the permission of the file /etc/pki/java/cacerts is "lrwxrwxrwx." (using ls -lht command) Any suggestion here would really help The logs seen is as follows: [2017-07-28T11:49:54,321][INFO][o.e.n.Node] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+DisableExplicitGC, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Djavax.net.ssl.trustStore=/etc/pki/java/cacerts, -Djavax.net.ssl.truststore.password=changeit, -Des.path.home=/usr/share/elasticsearch] [2017-07-28T11:50:00,948][ERROR][o.e.b.Bootstrap] Exception org.elasticsearch.ElasticsearchException: Failed to load plugin class [org.elasticsearch.xpack.XPackPlugin] at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:430) ~[elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:383) ~[elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.plugins.PluginsService.(PluginsService.java:139) ~[elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.node.Node.(Node.java:310) ~[elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.node.Node.(Node.java:242) ~[elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:232) ~[elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:232) ~[elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:350) [elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) [elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) [elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) [elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) [elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.cli.Command.main(Command.java:88) [elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) [elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) [elasticsearch-5.4.2.jar:5.4.2] Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0\_131] at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:419) ~[elasticsearch-5.4.2.jar:5.4.2] ... 14 more Caused by: org.elasticsearch.ElasticsearchException: failed to initialize a TrustManagerFactory at org.elasticsearch.xpack.ssl.StoreTrustConfig.createTrustManager(StoreTrustConfig.java:57) ~[?:?] at org.elasticsearch.xpack.ssl.SSLService.createSslContext(SSLService.java:378) ~[?:?] at org.elasticsearch.xpack.ssl.SSLService.loadSSLConfigurations(SSLService.java:401) ~[?:?] at org.elasticsearch.xpack.ssl.SSLService.(SSLService.java:79) ~[?:?] at org.elasticsearch.xpack.XPackPlugin.(XPackPlugin.java:205) ~[?:?] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0\_131] at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:419) ~[elasticsearch-5.4.2.jar:5.4.2] ... 14 more Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "/etc/pki/java/cacerts" "read") at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0\_131] at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0\_131] at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0\_131] at java.lang.SecurityManager.checkRead(SecurityManager.java:888) ~[?:1.8.0\_131] at sun.nio.fs.UnixChannelFactory.open(UnixChannelFactory.java:245) ~[?:?] at sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:136) ~[?:?] at sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:148) ~[?:?] at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:212) ~[?:?] at java.nio.file.Files.newByteChannel(Files.java:361) ~[?:1.8.0\_131] at java.nio.file.Files.newByteChannel(Files.java:407) ~[?:1.8.0\_131] at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384) ~[?:1.8.0\_131] at java.nio.file.Files.newInputStream(Files.java:152) ~[?:1.8.0\_131] at org.elasticsearch.xpack.ssl.CertUtils.trustManager(CertUtils.java:164) ~[?:?] at org.elasticsearch.xpack.ssl.StoreTrustConfig.createTrustManager(StoreTrustConfig.java:55) ~[?:?] at org.elasticsearch.xpack.ssl.SSLService.createSslContext(SSLService.java:378) ~[?:?] at org.elasticsearch.xpack.ssl.SSLService.loadSSLConfigurations(SSLService.java:401) ~[?:?] ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [July 28, 2017, 7:47am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/2 "2017-07-28T07:47:53Z")
If you are using ES 5.x, then `watcher.http.ssl.keystore` is the wrong setting, as those have been moved over to the `xpack` namespace, so it should be `xpack.http.ssl.keystore` However the exception stats something else in addition. ```auto Caused by: java.security.AccessControlException: access denied (“java.io.FilePermission” “/etc/pki/java/cacerts” “read”) ``` You need to put the certs into the elasticsearch configuration directory, you cannt put them anywhere in the filesystems. --Alex ---
**Author:** ![biswars](https://avatars.discourse-cdn.com/v4/letter/b/d9b06d/32.png) [@biswars](https://discuss.elastic.co/u/biswars)\ **Post date:** [July 28, 2017, 11:31am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/3 "2017-07-28T11:31:27Z")
Thanks @spinscale . The file permission error is resolved now. Just a query out of my curiosity as the elasticsearch is running as the user "elasticsearch" and the certs file is present in a path that has full access, how was it still not accessible !! I am yet to get over the "SSLHandshakeException[Received fatal alert: handshake\_failure". The relevant section in elasticsearch.yml looks like below xpack.http.ssl.certificate\_authorities: /etc/elasticsearch/cacert.pem (the CA cert) xpack.ssl.client\_authentication: none (Don't want to have the client cert presentation during the handshake) Anything missing here ? ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [July 28, 2017, 1:11pm UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/4 "2017-07-28T13:11:56Z")
please supply exceptions (also check the logs of the master node on startup), everything is else just guessing ---
**Author:** ![biswars](https://avatars.discourse-cdn.com/v4/letter/b/d9b06d/32.png) [@biswars](https://discuss.elastic.co/u/biswars)\ **Post date:** [July 28, 2017, 1:23pm UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/5 "2017-07-28T13:23:53Z")
The log shows the same exception "SSLHandshakeException" and I believe it is due to the CA cert is not trusted. I used the PEM encoded cert same as used in the target host (to which the web action is connecting to) setting it to the config param xpack.http.ssl.certificate\_authorities which did not work either. For reference, the following is the log in the master: [2017-07-28T18:22:51,797][ERROR][o.e.x.w.a.w.ExecutableWebhookAction] [A0cH-TU] failed to execute action [_inlined_/t\_webhook] javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake\_failure at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[?:?] at sun.security.ssl.Alerts.getSSLException(Alerts.java:154) ~[?:?] at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2033) ~[?:?] at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1135) ~[?:?] at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385) ~[?:?] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413) ~[?:?] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397) ~[?:?] at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) ~[?:?] at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) ~[?:?] at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1316) ~[?:?] at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1291) ~[?:?] at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250) ~[?:?] at org.elasticsearch.xpack.common.http.HttpClient.doExecute(HttpClient.java:171) ~[x-pack-5.4.2.jar:5.4.2] at org.elasticsearch.xpack.common.http.HttpClient.execute(HttpClient.java:92) ~[x-pack-5.4.2.jar:5.4.2] at org.elasticsearch.xpack.watcher.actions.webhook.ExecutableWebhookAction.execute(ExecutableWebhookAction.java:59) ~[x-pack-5.4.2.jar:5.4.2] at org.elasticsearch.xpack.watcher.actions.ActionWrapper.execute(ActionWrapper.java:160) [x-pack-5.4.2.jar:5.4.2] at org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:412) [x-pack-5.4.2.jar:5.4.2] at org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:275) [x-pack-5.4.2.jar:5.4.2] at org.elasticsearch.xpack.watcher.transport.actions.execute.TransportExecuteWatchAction.masterOperation(TransportExecuteWatchAction.java:143) [x-pack-5.4.2.jar:5.4.2] at org.elasticsearch.xpack.watcher.transport.actions.execute.TransportExecuteWatchAction.masterOperation(TransportExecuteWatchAction.java:65) [x-pack-5.4.2.jar:5.4.2] at org.elasticsearch.action.support.master.TransportMasterNodeAction.masterOperation(TransportMasterNodeAction.java:87) [elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$2.doRun(TransportMasterNodeAction.java:166) [elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.4.2.jar:5.4.2] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.4.2.jar:5.4.2] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0\_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0\_131] ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [August 1, 2017, 6:55am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/6 "2017-08-01T06:55:23Z")
Can you also share your `elasticsearch.yml` configuration, at least all the `xpack.` settings? Thanks! ---
**Author:** ![biswars](https://avatars.discourse-cdn.com/v4/letter/b/d9b06d/32.png) [@biswars](https://discuss.elastic.co/u/biswars)\ **Post date:** [August 1, 2017, 9:36am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/7 "2017-08-01T09:36:47Z")
Here it is: xpack.http.ssl.certificate\_authorities: /etc/elasticsearch/cacert.pem (the CA cert) xpack.ssl.client\_authentication: none (Don’t want to have the client cert presentation during the handshake) ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [August 4, 2017, 6:04am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/8 "2017-08-04T06:04:50Z")
Hey, I dug through the code and this is not supported currently. We need to add this though. --Alex ---
**Author:** ![biswars](https://avatars.discourse-cdn.com/v4/letter/b/d9b06d/32.png) [@biswars](https://discuss.elastic.co/u/biswars)\ **Post date:** [August 4, 2017, 8:43am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/9 "2017-08-04T08:43:38Z")
Thanks @spinscale for this confirmation. So by any other means we can get over this by importing the cert and using -Djavax.net.ssl.trustStore option during the jvm startup ? ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [August 4, 2017, 8:56am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/10 "2017-08-04T08:56:24Z")
yes. you need to set up your own truststore and use the `xpack.http.ssl` settings to point to it. If that one is not configured, the standard `ssl` truststore will be used. ---
**Author:** ![biswars](https://avatars.discourse-cdn.com/v4/letter/b/d9b06d/32.png) [@biswars](https://discuss.elastic.co/u/biswars)\ **Post date:** [August 4, 2017, 10:14am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/11 "2017-08-04T10:14:05Z")
Just to re-confirm, can I use xpack.http.ssl.keystore.path to point to my own truststore ? Secondly, by standard truststore do you mean using the -Djavax.net.ssl.trustStore option during the jvm startup ? ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [August 7, 2017, 7:07am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/12 "2017-08-07T07:07:53Z")
Exactly, you can point to your own truststore. Sorry for not being clear, with standard truststore I mean the one, you configure under the `xpack.ssl.truststore` configuration namespace, that works also for security and the realms. ---
**Author:** ![biswars](https://avatars.discourse-cdn.com/v4/letter/b/d9b06d/32.png) [@biswars](https://discuss.elastic.co/u/biswars)\ **Post date:** [August 9, 2017, 10:04am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/13 "2017-08-09T10:04:39Z")
I have tried using the configurations as mentioned above by setting xpack.ssl.truststore.path to the .jks file and xpack.ssl.truststore.password to the password of the truststore. However, I still get exactly the same error as earlier. Anything missing still ? Note: I have also tried using -Djavax.net.ssl.trustStore and -Djavax.net.ssl.trustStorePassword and the error received is still the same ---
**Author:** ![biswars](https://avatars.discourse-cdn.com/v4/letter/b/d9b06d/32.png) [@biswars](https://discuss.elastic.co/u/biswars)\ **Post date:** [August 29, 2017, 6:26am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/14 "2017-08-29T06:26:18Z")
Have tried the same trustostore with a custom java client (using SSLSocketFactory) as well and it works fine. Is it that the truststore is completely ignored by elasticsearch even using -Djavax.net.ssl.trustStore and -Djavax.net.ssl.trustStorePassword? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 26, 2017, 6:26am UTC](https://discuss.elastic.co/t/sslhandshakeexception-while-using-webhook-action-in-watcher/94906/15 "2017-09-26T06:26:59Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.