Watcher email configuration

Hi,

I need to enable the watcher email alert, I have added below setting in the elasticsearch.yml:

xpack.watcher.enabled: true

xpack.notification.email.account:

** gmail_account:**
** profile: gmail**
** smtp:**
** auth: true**
** starttls.enable: true**
** host: smtp.gmail.com**
** port: 587**
** user: "myuser@gmail.com"**

also have save my secure password in elasticsearch-keystore, PFB:

[root@myserver bin]# ./elasticsearch-keystore list
warning: ignoring JAVA_OPTS=-Xms256m -Xmx2048m; pass JVM parameters via ES_JAVA_OPTS
bootstrap.password
keystore.seed
xpack.notification.email.account.gmail_account.smtp.secure_password

But when I am trying to test the email service in the watcher, it produces the below exception:

[2019-11-18T00:20:12,782][ERROR][o.e.x.w.a.e.ExecutableEmailAction] [myserver] failed to execute action [inlined/email_1]
javax.mail.MessagingException: failed to send email with subject [Watch [abc] has exceeded the threshold] via account [gmail_account]

  •    at org.elasticsearch.xpack.watcher.notification.email.EmailService.send(EmailService.java:153) ~[?:?]*
    
  •    at org.elasticsearch.xpack.watcher.notification.email.EmailService.send(EmailService.java:145) ~[?:?]*
    
  •    at org.elasticsearch.xpack.watcher.actions.email.ExecutableEmailAction.execute(ExecutableEmailAction.java:72) ~[?:?]*
    
  •    at org.elasticsearch.xpack.core.watcher.actions.ActionWrapper.execute(ActionWrapper.java:144) [x-pack-core-7.1.1.jar:7.1.1]*
    
  •    at org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:516) [x-pack-watcher-7.1.1.jar:7.1.1]*
    
  •    at org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:309) [x-pack-watcher-7.1.1.jar:7.1.1]*
    
  •    at org.elasticsearch.xpack.watcher.transport.actions.execute.TransportExecuteWatchAction$1.doRun(TransportExecuteWatchAction.java:159) [x-pack-watcher-7.1.1.jar:7.1.1]*
    
  •    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.1.1.jar:7.1.1]*
    
  •    at org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:605) [x-pack-watcher-7.1.1.jar:7.1.1]*
    
  •    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:681) [elasticsearch-7.1.1.jar:7.1.1]*
    
  •    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]*
    
  •    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]*
    
  •    at java.lang.Thread.run(Thread.java:835) [?:?]*
    

Caused by: com.sun.mail.util.MailConnectException: Couldn't connect to host, port: smtp.gmail.com, 587; timeout 120000

  •    at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:2209) ~[?:?]*
    
  •    at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:740) ~[?:?]*
    
  •    at javax.mail.Service.connect(Service.java:366) ~[?:?]*
    
  •    at org.elasticsearch.xpack.watcher.notification.email.Account.lambda$executeConnect$2(Account.java:158) ~[?:?]*
    
  •    at java.security.AccessController.doPrivileged(AccessController.java:551) ~[?:?]*
    
  •    at org.elasticsearch.xpack.watcher.notification.email.Account.executeConnect(Account.java:157) ~[?:?]*
    
  •    at org.elasticsearch.xpack.watcher.notification.email.Account.send(Account.java:116) ~[?:?]*
    
  •    at org.elasticsearch.xpack.watcher.notification.email.EmailService.send(EmailService.java:151) ~[?:?]*
    
  •    ... 12 more*
    

When I have changed the configuration in yml as below:

xpack.watcher.enabled: true
xpack.notification.email.account:
** work:**
** profile: gmail**
** smtp:**
** auth: true**
** starttls.enable: true**
** host: smtp.gmail.com**
** port: 587**
** user: "myuser@gmail.com"**

Then it is reflecting that it couldn't find the password, and getting the exception as below:

[2019-11-18T18:03:17,790][ERROR][o.e.x.w.a.e.ExecutableEmailAction] [myserver] failed to execute action [inlined/email_1]
javax.mail.MessagingException: failed to send email with subject [Watch [abac] has exceeded the threshold] via account [work]

  •    at org.elasticsearch.xpack.watcher.notification.email.EmailService.send(EmailService.java:153) ~[?:?]*
    
  •    at org.elasticsearch.xpack.watcher.notification.email.EmailService.send(EmailService.java:145) ~[?:?]*
    
  •    at org.elasticsearch.xpack.watcher.actions.email.ExecutableEmailAction.execute(ExecutableEmailAction.java:72) ~[?:?]*
    
  •    at org.elasticsearch.xpack.core.watcher.actions.ActionWrapper.execute(ActionWrapper.java:144) [x-pack-core-7.1.1.jar:7.1.1]*
    
  •    at org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:516) [x-pack-watcher-7.1.1.jar:7.1.1]*
    
  •    at org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:309) [x-pack-watcher-7.1.1.jar:7.1.1]*
    
  •    at org.elasticsearch.xpack.watcher.transport.actions.execute.TransportExecuteWatchAction$1.doRun(TransportExecuteWatchAction.java:159) [x-pack-watcher-7.1.1.jar:7.1.1]*
    
  •    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.1.1.jar:7.1.1]*
    
  •    at org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:605) [x-pack-watcher-7.1.1.jar:7.1.1]*
    
  •    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:681) [elasticsearch-7.1.1.jar:7.1.1]*
    
  •    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]*
    
  •    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]*
    
  •    at java.lang.Thread.run(Thread.java:835) [?:?]*
    

Caused by: javax.mail.AuthenticationFailedException: failed to connect, no password specified?

  •    at javax.mail.Service.connect(Service.java:400) ~[?:?]*
    
  •    at org.elasticsearch.xpack.watcher.notification.email.Account.lambda$executeConnect$2(Account.java:158) ~[?:?]*
    
  •    at java.security.AccessController.doPrivileged(AccessController.java:551) ~[?:?]*
    
  •    at org.elasticsearch.xpack.watcher.notification.email.Account.executeConnect(Account.java:157) ~[?:?]*
    
  •    at org.elasticsearch.xpack.watcher.notification.email.Account.send(Account.java:116) ~[?:?]*
    
  •    at org.elasticsearch.xpack.watcher.notification.email.EmailService.send(EmailService.java:151) ~[?:?]*
    
  •    ... 12 more*
    

Requesting you to please suggest, what i am missing...... and where I am doing wrong.
Any help would be appreciable.

Thanks,
Faiz Ahmad

So, there are two different exceptions

Couldn't connect to host, port: smtp.gmail.com, 587; timeout 120000

This one means, that it could connect to gmail, but was not able to send an email within 2 minutes (which is a long time)

Caused by: javax.mail.AuthenticationFailedException: failed to connect, no password specified?

This one means that the user/password could not be successfully authenticated.

I have a hard time spotting the differences between the two setups, maybe you can explain this. Also make sure that after changing a keystore you either restart the node or reload secure settings. And make sure that setting is on every keystore on each node in the cluster (and reloaded), before trying further.