I am trying to create a watcher which generates daily/weekly reports of a dashboard and send that via emails. When the watcher is getting executed, it shows an SSL exception in the output (shown in the picture)
The problem happens when watcher attempts to download the file from https://xyz as it cannot validate the TLS certificate that is presented for https://xyz
No, not necessarily, I was just probing for what your setup looks like since that was not included in the original information.
The issue is as I describe above, and the generic solution is to make sure that watcher trusts your certificate or the certificate authority that has signed it . It is sadly not possible to give you a specific suggestion without knowing the rest of your configuration but note that Watcher ssl settings with regards to what certificates are trusted are described here https://www.elastic.co/guide/en/elasticsearch/reference/current/notification-settings.html#_watcher_tls_ssl_key_and_trusted_certificate_settings . If you haven't explicitly set that then it falls back to your node's SSL settings ( xpack.ssl.X) . If those haven't been set either, then it falls back to the truststore of your JVM ( the cacert file that ships with your java installation ).
If you can share some more details regarding
What kind of a certificate you're using for kibana ( you said it's not self signed ,but is it signed by a public CA or an internal one ? ) I'd you don't know and you have access to openssl binary in your environment you can run
openssl s_client -connect xyz:443
and share the output, or at least your gained insights from running it.
Your SSL configuration in elasticsearch.yml
we would be able to offer some more specific suggestions
@ikakavas I'm helping @swathi on this issue so might be able to help out some. The are using public CA for the SSL cert to the kibana UI. We are using self-signed certs for the node to node communications.
Here is our kibana reporting settings(i've removed the key but it's defined) :
This is the cause of your problem.
This tells X-Pack to only trust "certs-path" and not trust the JRE's defaults CAs.
It is almost always a mistake to configure xpack.ssl.truststore.path or xpack.ssl.certificate_authorities. (I accept that our docs used to recommend that you do this - they shouldn't have).
The settings below are all related to trust, and for HTTP that means they affect how PKI (client certificates) work, but don't affect the way SSL works for the server itself. Unless you are trying to configure a particular setup for PKI, you probably don't want to configure them for HTTP (but you typically do want them for transport).
Do you have any more details on the exception - it's possible that it's coming from your SMTP server rather than Kibana. Are you accessing the email server over SSL?
This was my original thinking since the error was inside the email block of the script. However in other watcher's I am able to send simple emails, without attached reports. I've also verified that I'm not using SSL to the mail server.
Here is my settings for the mail server:
xpack.notification.email.account:
smtp_account:
profile: standard
email_defaults:
from: svcs@xyz.com
smtp:
auth: false
starttls.enable: false
host: mailhost.xyz.com
port: 25
Also, I have emails being sent from other watches that are delivered properly. Wouldn't a misconfigured email certificate give the same error in that case?
There are a few certs in play here. The first is for node to node elasticsearch communication, and it is all self-signed. Next is our private CA signed cert for https connections via nginx or kibana. The last is for the public https URL metrics.xyz.com and it is signed by a public CA (this terminates on a load balancer).
I had added the truststore on the coordinator nodes only, but noticed i the logs, that the watches were being run randomly from the data nodes or coordinator nodes. So I added the truststore to all of them, and this has solved the issue. Thanks!
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.