Watcher for report with attachment failed due to ssl handshake exception

Elastic Stack Kibana
,
1 
Unable to send reporting attachment by the help of watcher. My watch query : 

PUT _watcher/watch/sample_report_ssl
{
  "trigger" : {
    "schedule": {
      "interval": "1m"
    }
  },
  "actions" : {
    "email_admin" : { 
      "email": {
        "from": "*******@gmail.com",
        "to": "*********@gmail.com",
        "subject": "Error Report",
        "attachments" : {
          "error_report.pdf" : {
            "reporting" : {
              "url": "Copied Post URL from Share tab", 
              "retries":40, 
              "interval":"15s", 
              "auth":{ 
                "basic":{
                  "username":"abc",
                  "password":"******"
                }
              }
            }
          }
        }
      }
    }
  }
}

resulting exception:

 "actions": [
      {
        "id": "email_admin",
        "type": "email",
        "status": "failure",
        "error": {
          "root_cause": [
            {
              "type": "s_s_l_handshake_exception",
              "reason": "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"
            }
          ],
          "type": "s_s_l_handshake_exception",
          "reason": "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
          "caused_by": {
            "type": "validator_exception",
            "reason": "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
            "caused_by": {
              "type": "sun_cert_path_builder_exception",
              "reason": "unable to find valid certification path to requested target"


My elasticsearch configuration looks like : 

xpack.security.http.ssl.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.key: /etc/elasticsearch/certs/node-1.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/node-1.crt
xpack.security.http.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
xpack.security.transport.ssl.key: /etc/elasticsearch/certs/node-1.key
xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/node-1.crt
xpack.security.transport.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
#########################################################
xpack.http.ssl.truststore.path: /etc/elasticsearch/certs/watcher-truststore.jks
xpack.http.ssl.truststore.password: ******
xpack.security.audit.enabled: true
xpack.watcher.enabled: true

Kibana Configuration is : 

xpack.reporting.enabled: true
#watcher setting for SSL/TLS based reporting
server.ssl.enabled: true
#elasticsearch.ssl.verificationMode: certificate
server.ssl.certificate: "/etc/kibana/config/certs/kibana-1.crt"
server.ssl.key: "/etc/kibana/config/certs/kibana-1.key"
elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/config/certs/ca.crt"]
elasticsearch.ssl.verificationMode: certificate

I am following this article : https://www.elastic.co/guide/en/kibana/7.3/securing-reporting.html

Please suggest, thanking you in advance.

1 Like
2

As you use self-signed certificate in kibana, you need to add the CA certificate used to generate the kibana certificate for by watcher for https connection to kibana reporting in elasticsearch.yml
In this case this is setting called xpack.http.ssl.certificate_authorities which is currently missing or ensure that ca.crt for kibana certicate is included in /etc/elasticsearch/certs/watcher-truststore.jks

1 Like
3 
Hi Julien, 
Thanks for your reply. For kibana, i have below mentioned self sign certificate, key and CA certificate available : 
(a) /etc/kibana/config/certs/kibana-1.crt
(b) /etc/kibana/config/certs/kibana-1.key
(c) /etc/kibana/config/certs/ca.crt**

Now as per your advice,if i am adding  , xpack.http.ssl.certificate_autorities, in my elasticsearch.yml file like this:

xpack.security.http.ssl.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.key: /etc/elasticsearch/certs/node-1.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/node-1.crt
xpack.security.http.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
xpack.security.transport.ssl.key: /etc/elasticsearch/certs/node-1.key
xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/node-1.crt
xpack.security.transport.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
xpack.http.ssl.certificate_authorities: [ "/etc/kibana/config/certs/ca.cert" ]
#########################################################
xpack.http.ssl.truststore.path: /etc/elasticsearch/certs/watcher-truststore.jks
xpack.http.ssl.truststore.password: *****

unable to start the elasticsearch, for generating watcher-truststore.jks i have used /etc/kibana/config/certs/kibana-1.crt certificate, it is giving below mentioned error:

[2020-06-29T01:31:59,137][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-1] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: ElasticsearchSecurityException[failed to load SSL configuration [xpack.http.ssl]]; nested: IllegalArgumentException[you cannot specify a truststore and ca files];
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:174) ~[elasticsearch-7.6.0.jar:7.6.0]
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.6.0.jar:7.6.0]
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.6.0.jar:7.6.0]
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.6.0.jar:7.6.0]
	at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.6.0.jar:7.6.0]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.6.0.jar:7.6.0]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.6.0.jar:7.6.0]
Caused by: org.elasticsearch.ElasticsearchSecurityException: failed to load SSL configuration [xpack.http.ssl]
	at org.elasticsearch.xpack.core.ssl.SSLService.loadConfiguration(SSLService.java:524) ~[?:?]
	at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$4(SSLService.java:497) ~[?:?]
	at java.util.HashMap.forEach(HashMap.java:1338) ~[?:?]
	at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:497) ~[?:?]
	at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:142) ~[?:?]
	at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:130) ~[?:?]
	at org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:259) ~[?:?]
	at org.elasticsearch.node.Node.lambda$new$9(Node.java:456) ~[elasticsearch-7.6.0.jar:7.6.0]
	at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1621) ~[?:?]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
	at org.elasticsearch.node.Node.<init>(Node.java:459) ~[elasticsearch-7.6.0.jar:7.6.0]
	at org.elasticsearch.node.Node.<init>(Node.java:257) ~[elasticsearch-7.6.0.jar:7.6.0]
	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) ~[elasticsearch-7.6.0.jar:7.6.0]
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.6.0.jar:7.6.0]
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.6.0.jar:7.6.0]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.6.0.jar:7.6.0]
	... 6 more
Caused by: java.lang.IllegalArgumentException: you cannot specify a truststore and ca files
	at org.elasticsearch.xpack.core.ssl.SSLConfiguration.createCertChainTrustConfig(SSLConfiguration.java:181) ~[?:?]
	at org.elasticsearch.xpack.core.ssl.SSLConfiguration.createTrustConfig(SSLConfiguration.java:170) ~[?:?]
	at org.elasticsearch.xpack.core.ssl.SSLConfiguration.<init>(SSLConfiguration.java:50) ~[?:?]
	at org.elasticsearch.xpack.core.ssl.SSLService.loadConfiguration(SSLService.java:519) ~[?:?]
	at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$4(SSLService.java:497) ~[?:?]
	at java.util.HashMap.forEach(HashMap.java:1338) ~[?:?]

so , as per error, i have changed the elasticsearch.yml configuration as:

xpack.security.http.ssl.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.key: /etc/elasticsearch/certs/node-1.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/node-1.crt
xpack.security.http.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
xpack.security.transport.ssl.key: /etc/elasticsearch/certs/node-1.key
xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/node-1.crt
xpack.security.transport.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
xpack.http.ssl.certificate_authorities: [ "/etc/kibana/config/certs/ca.cert" ]

Now it is giving below mentioned error:

Caused by: org.elasticsearch.ElasticsearchException: failed to initialize SSL TrustManager - access to read certificate_authorities file [/etc/kibana/config/certs/ca.cert] is blocked; SSL resources should be placed in the [/etc/elasticsearch] directory
	at org.elasticsearch.xpack.core.ssl.TrustConfig.blockedTrustConfigFile(TrustConfig.java:135) ~[?:?]
	at org.elasticsearch.xpack.core.ssl.PEMTrustConfig.createTrustManager(PEMTrustConfig.java:61) ~[?:?]
	at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:427) ~[?:?]
	at java.util.HashMap.computeIfAbsent(HashMap.java:1138) ~[?:?]
	at org.elasticsearch.xpack.core.ssl.SSLService.loadConfiguration(SSLService.java:521) ~[?:?]
	... 26 more
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "/etc/kibana/config/certs/ca.cert" "read")
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
	at java.security.AccessController.checkPermission(AccessController.java:1036) ~[?:?]
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:408) ~[?:?]
	at java.lang.SecurityManager.checkRead(SecurityManager.java:747) ~[?:?]
	at sun.nio.fs.UnixChannelFactory.open(UnixChannelFactory.java:255) ~[?:?]
	at sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:143) ~[?:?]
	at sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:156) ~[?:?]
	at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:217) ~[?:?]
	at java.nio.file.Files.newByteChannel(Files.java:374) ~[?:?]
	at java.nio.file.Files.newByteChannel(Files.java:425) ~[?:?]

by checking this error come to know that, /etc/kibana/config/certs/ca.crt having permission issue, and suggesting that we need to place the file at /etc/elasticsearch location, so i have placed the file ca.crt file at this location using below mentioned command:

ubuntu@ip-172-31-13-56:~$ cp /etc/kibana/config/certs/ca.crt /etc/elasticsearch/
ubuntu@ip-172-31-13-56:/etc/elasticsearch$ ll ca.crt
-rwxrwxr-x 1 ubuntu elasticsearch 1200 Jun 29 06:26 ca.crt*

Now again i have changed the configuration of elasticsearch.yml file as:

xpack.security.http.ssl.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.key: /etc/elasticsearch/certs/node-1.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/node-1.crt
xpack.security.http.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
xpack.security.transport.ssl.key: /etc/elasticsearch/certs/node-1.key
xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/node-1.crt
xpack.security.transport.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
xpack.http.ssl.certificate_authorities: [ "/etc/elasticsearch/ca.crt" ]

And, now it is working. Thnx so much...........:slight_smile:

1 Like
4

Please ignore spelling mistake in configuration, that was blunder........:stuck_out_tongue:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.