# Watcher for Reporting shows an SSL Exception in the output

**URL:** <https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048>\
**Category:** Kibana\
**Tags:** elastic-stack-reporting\
**Created:** [June 30, 2018, 2:48am UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048 "2018-06-30T02:48:57Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![swathi](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@swathi](https://discuss.elastic.co/u/swathi)\
**Post date:** [June 30, 2018, 2:48am UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/1 "2018-06-30T02:48:57Z")

</div>

Hello,

I am trying to create a watcher which generates daily/weekly reports of a dashboard and send that via emails. When the watcher is getting executed, it shows an SSL exception in the output (shown in the picture)

 ![31%20PM](https://us1.discourse-cdn.com/elastic/original/3X/9/0/90ee9ccc3efdfc87e580514531b71ebf98e98231.png)

Below is the watcher I am using for this:

```
{
  "trigger": {
    "schedule": {
      "interval": "30s"
    }
  },
  "input": {
    "none": {}
  },
  "condition": {
    "always": {}
  },
  "actions": {
    "email_admin": {
      "email": {
        "profile": "standard",
        "attachments": {
          "error.pdf": {
            "reporting": {
              "url": "https://xyz",
              "retries": 5,
              "interval": "15s"
            }
          }
        },
        "to": [
          "myemail@mydomain.com"
        ],
        "subject": "Weekly Monitoring Report"
      }
    }
  }
}

```

Please help me understand why this error is occurring here and how to solve this. Thanks in advance.

---

<div class="post-metadata">

**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\
**Post date:** [July 2, 2018, 11:38am UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/2 "2018-07-02T11:38:52Z")

</div>

Hi,

The problem happens when watcher attempts to download the file from [https://xyz](https://xyz) as it cannot validate the TLS certificate that is presented for [https://xyz](https://xyz)

I am assuming the URL is pointing to your Kibana instance. What are your SSL settings in Kibana? Are you using a self signed certificate for TLS in Kibana ? ( Step 1 in this [https://www.elastic.co/guide/en/kibana/6.3/configuring-tls.html#configuring-tls](https://www.elastic.co/guide/en/kibana/6.3/configuring-tls.html#configuring-tls) )

---

<div class="post-metadata">

**Author:** ![swathi](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@swathi](https://discuss.elastic.co/u/swathi)\
**Post date:** [July 3, 2018, 4:59pm UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/3 "2018-07-03T16:59:05Z")

</div>

@ikakavas no, I am not using any self-signed certificate. Could that cause this problem?

---

<div class="post-metadata">

**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\
**Post date:** [July 3, 2018, 5:26pm UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/4 "2018-07-03T17:26:15Z")

</div>

No, not necessarily, I was just probing for what your setup looks like since that was not included in the original information.

The issue is as I describe above, and the generic solution is to make sure that watcher trusts your certificate or the certificate authority that has signed it . It is sadly not possible to give you a specific suggestion without knowing the rest of your configuration but note that Watcher ssl settings with regards to what certificates are trusted are described here [https://www.elastic.co/guide/en/elasticsearch/reference/current/notification-settings.html#\_watcher\_tls\_ssl\_key\_and\_trusted\_certificate\_settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/notification-settings.html#_watcher_tls_ssl_key_and_trusted_certificate_settings) . If you haven't explicitly set that then it falls back to your node's SSL settings ( `xpack.ssl.X`) . If those haven't been set either, then it falls back to the truststore of your JVM ( the cacert file that ships with your java installation ).

If you can share some more details regarding

1. What kind of a certificate you're using for kibana ( you said it's not self signed ,but is it signed by a public CA or an internal one ? ) I'd you don't know and you have access to openssl binary in your environment you can run

```auto
openssl s_client -connect xyz:443

```

and share the output, or at least your gained insights from running it.

1. Your SSL configuration in elasticsearch.yml

we would be able to offer some more specific suggestions

---

<div class="post-metadata">

**Author:** ![swathi](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@swathi](https://discuss.elastic.co/u/swathi)\
**Post date:** [July 3, 2018, 5:44pm UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/5 "2018-07-03T17:44:18Z")

</div>

Thanks for responding.

1. I am using a public CA certificate.
2. My SSL configuration in elasticsearch.yml is below:

> ```
> xpack.ssl.keystore.path: certs-path
> xpack.ssl.truststore.path: certs-path
> xpack.security.transport.ssl.enabled: true
> xpack.security.transport.ssl.verification_mode: certificate
> xpack.security.http.ssl.enabled: true
> xpack.security.http.ssl.verification_mode: certificate
> 
> ```

---

<div class="post-metadata">

**Author:** ![jspeer](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jspeer](https://discuss.elastic.co/u/jspeer)\
**Post date:** [July 3, 2018, 9:17pm UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/6 "2018-07-03T21:17:33Z")

</div>

@ikakavas I'm helping @swathi on this issue so might be able to help out some. The are using public CA for the SSL cert to the kibana UI. We are using self-signed certs for the node to node communications.

Here is our kibana reporting settings(i've removed the key but it's defined) :

xpack.reporting.enabled: true  
xpack.reporting.encryptionKey:  
xpack.reporting.kibanaServer.port: 443  
xpack.reporting.kibanaServer.protocol: https  
xpack.reporting.kibanaServer.hostname: [metrics.xyz.com](http://metrics.xyz.com)

---

<div class="post-metadata">

**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\
**Post date:** [July 3, 2018, 11:49pm UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/7 "2018-07-03T23:49:12Z")

</div>

> [@swathi](#):
>
> ```
> xpack.ssl.truststore.path: certs-path
> 
> ```

This is the cause of your problem.  
This tells X-Pack to only trust "certs-path" and not trust the JRE's defaults CAs.  
It is almost always a mistake to configure `xpack.ssl.truststore.path` or `xpack.ssl.certificate_authorities`. (I accept that our docs used to recommend that you do this - they shouldn't have).

You should instead set your config to be:

```auto
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.keystore.path: certs-path
xpack.security.transport.ssl.truststore.path: certs-path
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs-path

```

You almost certainly don't want to have this setting:

```auto
xpack.security.http.ssl.verification_mode: certificate

```

The settings below are all related to _trust_, and for HTTP that means they affect how PKI (client certificates) work, but don't affect the way SSL works for the server itself. Unless you are trying to configure a particular setup for PKI, you probably don't want to configure them for HTTP (but you typically do want them for transport).

- `verification_mode`
- `truststore.path`
- `certificate_authorities`

---

<div class="post-metadata">

**Author:** ![jspeer](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jspeer](https://discuss.elastic.co/u/jspeer)\
**Post date:** [July 16, 2018, 6:01pm UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/8 "2018-07-16T18:01:33Z")

</div>

OK, I've made these change to the config, however I'm still receiving the same Error when I try to execute the watch.

---

<div class="post-metadata">

**Author:** ![jspeer](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jspeer](https://discuss.elastic.co/u/jspeer)\
**Post date:** [July 16, 2018, 6:06pm UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/9 "2018-07-16T18:06:40Z")

</div>

`xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.keystore.path: certs-path xpack.security.transport.ssl.truststore.path: certs-path xpack.security.transport.ssl.verification_mode: certificate xpack.security.http.ssl.enabled: true xpack.security.http.ssl.keystore.path: certs-path`

---

<div class="post-metadata">

**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\
**Post date:** [July 17, 2018, 7:18am UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/10 "2018-07-17T07:18:59Z")

</div>

Do you have any more details on the exception - it's possible that it's coming from your SMTP server rather than Kibana. Are you accessing the email server over SSL?

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [July 17, 2018, 7:22am UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/11 "2018-07-17T07:22:12Z")

</div>

can you also please share the full output of the [execute watch API](https://www.elastic.co/guide/en/elasticsearch/reference/6.3/watcher-api-execute-watch.html) or a full watch history entry? Thanks a lot!

---

<div class="post-metadata">

**Author:** ![jspeer](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jspeer](https://discuss.elastic.co/u/jspeer)\
**Post date:** [July 17, 2018, 3:22pm UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/12 "2018-07-17T15:22:54Z")

</div>

This was my original thinking since the error was inside the email block of the script. However in other watcher's I am able to send simple emails, without attached reports. I've also verified that I'm not using SSL to the mail server.

---

<div class="post-metadata">

**Author:** ![jspeer](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jspeer](https://discuss.elastic.co/u/jspeer)\
**Post date:** [July 17, 2018, 3:27pm UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/13 "2018-07-17T15:27:25Z")

</div>

Here is the output of a test execute. Thanks!

```
{
"watch_id": "_inlined_",
"node": "JBNgvNcpR_COY5YvJNncbQ",
"state": "executed",
"status": {
  "state": {
    "active": true,
    "timestamp": "2018-07-17T15:23:15.102Z"
  },
  "last_checked": "2018-07-17T15:23:15.103Z",
  "last_met_condition": "2018-07-17T15:23:15.103Z",
  "actions": {
    "email_admin": {
      "ack": {
        "timestamp": "2018-07-17T15:23:15.102Z",
        "state": "awaits_successful_execution"
      },
      "last_execution": {
        "timestamp": "2018-07-17T15:23:15.103Z",
        "successful": false,
        "reason": ""
      }
    }
},
"execution_state": "executed",
"version": -1
},
"trigger_event": {
"type": "manual",
"triggered_time": "2018-07-17T15:23:15.103Z",
"manual": {
  "schedule": {
    "scheduled_time": "2018-07-17T15:23:15.103Z"
  }
}
},
"input": {
"none": {}
 },
"condition": {
  "always": {}
},
"metadata": {
  "xpack": {
    "type": "json"
  }
},
"result": {
  "execution_time": "2018-07-17T15:23:15.103Z",
  "execution_duration": 7,
  "input": {
    "type": "none",
    "status": "success",
    "payload": {}
  },
  "condition": {
    "type": "always",
    "status": "success",
    "met": true
  },
  "actions": [
  {
    "id": "email_admin",
    "type": "email",
    "status": "failure",
    "error": {
      "root_cause": [
        {
          "type": "s_s_l_handshake_exception",
          "reason": "sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"
        }
      ],
      "type": "s_s_l_handshake_exception",
      "reason": "sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
      "caused_by": {
        "type": "validator_exception",
        "reason": "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
        "caused_by": {
          "type": "sun_cert_path_builder_exception",
          "reason": "unable to find valid certification path to requested target"
        }
      }
    }
  }
]
},
"messages": []
}
```

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [July 18, 2018, 7:54am UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/14 "2018-07-18T07:54:29Z")

</div>

@jspeer as Tim already asked: Is it possible that you are using custom certificates with your mail server?

---

<div class="post-metadata">

**Author:** ![jspeer](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jspeer](https://discuss.elastic.co/u/jspeer)\
**Post date:** [July 18, 2018, 3:10pm UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/15 "2018-07-18T15:10:19Z")

</div>

Here is my settings for the mail server:  
xpack.notification.email.account:  
smtp\_account:  
profile: standard  
email\_defaults:  
from: [svcs@xyz.com](mailto:svcs@xyz.com)  
smtp:  
auth: false  
starttls.enable: false  
host: [mailhost.xyz.com](http://mailhost.xyz.com)  
port: 25

Also, I have emails being sent from other watches that are delivered properly. Wouldn't a misconfigured email certificate give the same error in that case?

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [July 19, 2018, 7:18am UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/16 "2018-07-19T07:18:06Z")

</div>

Hey,

indeed it would. Are you using custom certificates for TLS communication in Kibana as well? I assume that accessing kibana is causing this.

Can you remove the reporting attachment from that watch to verify if this is the case?

--Alex

---

<div class="post-metadata">

**Author:** ![jspeer](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jspeer](https://discuss.elastic.co/u/jspeer)\
**Post date:** [July 19, 2018, 4:57pm UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/17 "2018-07-19T16:57:40Z")

</div>

I've removed the report section, and the email sends without a problem. So there is something about this portion:

`"attachments": { "error.pdf": { "reporting": { "url": "https://vccn7:5601/api/reporting/jobs/download/jj0j4zsp0zw290eb70b9g26e", "retries": 5, "interval": "15s" } } },`

There are a few certs in play here. The first is for node to node elasticsearch communication, and it is all self-signed. Next is our private CA signed cert for https connections via nginx or kibana. The last is for the public https URL [metrics.xyz.com](http://metrics.xyz.com) and it is signed by a public CA (this terminates on a load balancer).

I've replaced the URL inside the watcher with all 3 possiblities for the URL:  
kibana: [https://vccn7:5601/api/reporting/jobs/download/jj0j4zsp0zw290eb70b9g26e](https://vccn7:5601/api/reporting/jobs/download/jj0j4zsp0zw290eb70b9g26e)  
nginx: [https://vccn7/api/reporting/jobs/download/jj0j4zsp0zw290eb70b9g26e](https://vccn7/api/reporting/jobs/download/jj0j4zsp0zw290eb70b9g26e)  
LB: [https://metrics.xyz.com/api/reporting/jobs/download/jj0j4zsp0zw290eb70b9g26e](https://metrics.xyz.com/api/reporting/jobs/download/jj0j4zsp0zw290eb70b9g26e)

However it's still not working.

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [July 20, 2018, 8:12am UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/18 "2018-07-20T08:12:55Z")

</div>

Hey,

so this indicates, that the cert you are using in kibana is not known on the elasticsearch side of things. Is this possible?

You can also configure an own truststore for the httpclient that is used by watcher to connect to kibana for example.

```auto
xpack.http.ssl.truststore.path: ...

```

--Alex

---

<div class="post-metadata">

**Author:** ![jspeer](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jspeer](https://discuss.elastic.co/u/jspeer)\
**Post date:** [August 13, 2018, 5:57pm UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/19 "2018-08-13T17:57:16Z")

</div>

I had added the truststore on the coordinator nodes only, but noticed i the logs, that the watches were being run randomly from the data nodes or coordinator nodes. So I added the truststore to all of them, and this has solved the issue. Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 10, 2018, 5:57pm UTC](https://discuss.elastic.co/t/watcher-for-reporting-shows-an-ssl-exception-in-the-output/138048/20 "2018-09-10T17:57:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.