Anyone one Could you please help me with the watcher configuration elasticsearch yml setup as we have 13 nodes I need to add the watcher settings to send a mail alert (outlook). Challenges what am facing here. It is completely secured project and need to add watcher settings to get email alert to my client. So I referred many pages in elastic I was facing issue adding setup in Elasticsearch yml file.
Elasticsearch version 7.17 version we are using
Totally 13 nodes ( master node - 3 , Data node - 10)
Questions — is it possible to test the watcher in one master node and one data node to enable the email alert ???
And In our environment we have default port 25 smtp
You'll need to add the settings to all your nodes for it to start working. That can be a big ask so often I'll save those changes for when I'm going to do a rolling restart anyway. Here is the documentation that should help out with watcher configuration.
Thanks for your support. I had configured the settings for outlook but password am missing to store in secure way. I didn’t added password in the watcher settings. How to proceed further. I need to add the password in all the nodes to move head. Till that watcher is disabled in kibana I can’t enable no option .
Could you please suggest how to add my outlook password to test the watcher email is working ??
Want to correct something from first reply, you'll need to add those changes to all the nodes, but I don't think you'll need to do a restart for the email settings. If you follow the link to the documentation provided on my original reply you'll see the following:
smtp.secure_password is a secure and reloadable setting.
Almost all the smtp settings are also dynamic, which means you can set them through Kibana.
As far as configuring it securely, see here for using secure settings.
This is what I'd use regarding watcher in your elasticsearch.yml file:
xpack.watcher.enabled: true
From dev tools in kibana I'd set the following cluster settings since they are dynamic:
xpack.notification.email.account.outlook_account.profile: outlook
xpack.notification.email.account.outlook_account.smtp.auth: true
xpack.notification.email.account.outlook_account.smtp.starttls.enable: true
xpack.notification.email.account.outlook_account.smtp.host: smtp-mail.outlook.com
xpack.notification.email.account.outlook_account.smtp.user: xxxxx@company.com
You could totally have them display nested didn't do that here.
You will then HAVE to add the setting "xpack.notification.email.account.outlook_account.smtp.secure_password" to the elasticsearch keystore for things to work securely. If that isn't added there it's never going to work. Please see the following documentation about doing that: elasticsearch-keystore docs.
Thanks a lot for ur help . Challenges what am facing adding outlook password in yml file.
I am not sure how to add password in Keystore I went through documentation but still I have concern regarding those things.
Example : In kibana under stack management- watcher- threshold alert - action - email ——-this is not enable yet. I tried adding watcher settings in yml file on 3 master nodes and restarted the elasticsearch still I don’t see the enable in kibana. Also lot of security concern I have to go step by step as explained earlier . We have 10 data nodes and 3 master nodes . In all these nodes I need to add watcher outlook settings and restart the service ???
Please read my previous posts because I feel like I've already answered these questions, but just to be clear:
Yes, you'll need to enable watcher and do a rolling-restart.
xpack.watcher.enabled: true
on all the nodes (master, data, etc). Just doing some will NOT work.
Can you tell me what your security concerns are? Without more information storing sensitive information in the elastic keystore is the best practice for handling that.
Ok, lets take a step back here and talk about what you're trying to accomplish. Reading between the lines it sounds like you might be connecting a fully functioning email address up with watcher. While that would probably work often a noreply address is used instead (noreply@example.com). Using an account like could limit the exposure if somehow the username and password were compromised.
Another option is if you just need some kind of alert, watcher supports other actions besides email. For example: webhook, slack, pagerduty. Perhaps one of those would work just as well.
Finally lets talk about the options to set variables in elastic.
You could set the password via command line. This is not secure at all because any other user logged into the system could see the password.
Setting the password directly in the elasticsearch.yml file. This is a little bit better assuming you have your file permissions set correctly, but you are still dependent upon the OS properly limiting read access to the yml file.
Using the keystore. The keystore was made for just these purposes. It will keep secure settings encrypted until needed by the program. This is the most secure option of the three.
To answer your questions about the keystore, no it is not mandatory could use option 1 or 2 if you wanted to but those aren't better. The keystore should be made automatically by elastic and should be in the same directory as your elasticsearch.yml file.
Final thought is please read and become familiar with the links provided. The answers to many of your questions will be found in the documentation, and while I try my best to answer and help this community the ultimate responsibility for your cluster falls on you and your understanding of the issues. Having said that I'm happy to help where I can, but can only do so much.
You need to configure the watcher settings in all the nodes in the cluster. Just master nodes won't cut it. Must enable on data nodes as well. You probably won't see an enable option in kibana, but you will be able to send emails from watcher once correctly configured.
I believe you won't have to put the password part in the elasticsearch.yml file because it will look in the keystore for it. You will need make sure the password is in each keystore as well.
Appreciate your support. I am working on data nodes to add the watcher setting remaining 10 nodes. If I face any issue challenging will let you know by tomorrow.
I have restarted all the data nodes and master nodes but the don’t see watcher enable in kibana.
Stack management - Watcher- create threshold alert- Add action - here I don’t see enable still.
Could you please help me out.
Also on see log message nohup.error. With 0 messages in all the nodes . But Elasticsearch is running fine .
Are you able to click the "Add Action" button? or is it hidden? If you can click the button which Actions do you see? For example in my 7.x cluster I see: Email, Logging, Slack, Webhook, Index, Pagerduty, Jira. As far as the nohup error. If it is still causing a problem I'd suggest opening up a different question for that.
I am able to configure watcher and enable email notification from kibana but facing issue in action tab —. Email- —- while testing the mail - we are getting error as “Failed to send email notification.”
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.