Watcher alert email action is not working

George_Lee · November 20, 2021, 6:45am

Deal all,

ECK version 1.7.1
Elasticsearch version 7.10.1

My Elasticsearch configuration is

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: elastic005
  namespace: elastic-system
spec:
  version: 7.10.1
  http:
    tls:
      selfSignedCertificate:
        disabled: true
  nodeSets:
  - name: default
    count: 1
    config:
      node.master: true
      node.data: true
      node.ingest: true
      node.store.allow_mmap: false
      xpack.notification.email.account.exchange_account.email_defaults.from: "***@***"
      xpack.notification.email.account.exchange_account.profile: "outlook"
      xpack.notification.email.account.exchange_account.smtp.auth: "true"
      xpack.notification.email.account.exchange_account.smtp.host: "***@***"
      xpack.notification.email.account.exchange_account.smtp.user: "***@***"
      xpack.notification.email.account.exchange_account.smtp.port: "25"
      xpack.notification.email.account.exchange_account.smtp.starttls.enable: "false"
      xpack.notification.email.default_account: "exchange_account"
  secureSettings:
  - secretName: elasticsearch-email-password

ECK helped me generate a single node es cluster，then I got the configuration is

cluster:
  name: elastic005
  routing:
    allocation:
      awareness:
        attributes: k8s_node_name
discovery:
  seed_providers: file
http:
  publish_host: ${POD_NAME}.${HEADLESS_SERVICE_NAME}.${NAMESPACE}.svc
network:
  host: "0"
  publish_host: ${POD_IP}
node:
  attr:
    k8s_node_name: ${NODE_NAME}
  data: true
  ingest: true
  master: true
  name: ${POD_NAME}
  store:
    allow_mmap: false
path:
  data: /usr/share/elasticsearch/data
  logs: /usr/share/elasticsearch/logs
xpack:
  license:
    upload:
      types:
      - trial
      - enterprise
  notification:
    email:
      account:
        exchange_account:
          email_defaults:
            from: ***@***
          profile: outlook
          smtp:
            auth: true
            host: ****
            port: 25
            starttls:
              enable: false
            user: ****
      default_account: exchange_account
  security:
    authc:
      realms:
        file:
          file1:
            order: -100
        native:
          native1:
            order: -99
      reserved_realm:
        enabled: "false"
    enabled: "true"
    http:
      ssl:
        certificate: /usr/share/elasticsearch/config/http-certs/tls.crt
        certificate_authorities: /usr/share/elasticsearch/config/http-certs/ca.crt
        enabled: false
        key: /usr/share/elasticsearch/config/http-certs/tls.key
    transport:
      ssl:
        certificate: /usr/share/elasticsearch/config/node-transport-cert/transport.tls.crt
        certificate_authorities:
        - /usr/share/elasticsearch/config/transport-certs/ca.crt
        - /usr/share/elasticsearch/config/transport-remote-certs/ca.crt
        enabled: "true"
        key: /usr/share/elasticsearch/config/node-transport-cert/transport.tls.key
        verification_mode: certificate

I tested a watcher alert and send mail, then i got an error

{"type": "server", "timestamp": "2021-11-20T06:38:44,096Z", "level": "ERROR", "component": "o.e.x.w.a.e.ExecutableEmailAction", "cluster.name": "elastic005", "node.name": "elastic005-es-default-0", "message": "failed to execute action [_inlined_/email_1]", "cluster.uuid": "I8-ddxlZTbeDrU7JErk_dg", "node.id": "X0_RjyybQ86fcOnDzlxRqw" ,
"stacktrace": ["javax.mail.MessagingException: failed to send email with subject [Watch [test] has exceeded the threshold] via account [exchange_account]",
"at org.elasticsearch.xpack.watcher.notification.email.EmailService.send(EmailService.java:171) ~[?:?]",
"at org.elasticsearch.xpack.watcher.notification.email.EmailService.send(EmailService.java:163) ~[?:?]",
"at org.elasticsearch.xpack.watcher.actions.email.ExecutableEmailAction.execute(ExecutableEmailAction.java:82) ~[?:?]",
"at org.elasticsearch.xpack.core.watcher.actions.ActionWrapper.execute(ActionWrapper.java:164) [x-pack-core-7.10.1.jar:?]",
"at org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:533) [x-pack-watcher-7.10.1.jar:7.10.1]",
"at org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:319) [x-pack-watcher-7.10.1.jar:7.10.1]",
"at org.elasticsearch.xpack.watcher.transport.actions.execute.TransportExecuteWatchAction$1.doRun(TransportExecuteWatchAction.java:159) [x-pack-watcher-7.10.1.jar:7.10.1]",
"at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.10.1.jar:7.10.1]",
"at org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:626) [x-pack-watcher-7.10.1.jar:7.10.1]",
"at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:678) [elasticsearch-7.10.1.jar:7.10.1]",
"at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]",
"at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]",
"at java.lang.Thread.run(Thread.java:832) [?:?]",
"Caused by: javax.mail.MessagingException: 501 5.5.4 Invalid domain name",
"",
"at com.sun.mail.smtp.SMTPTransport.issueCommand(SMTPTransport.java:2339) ~[?:?]",
"at com.sun.mail.smtp.SMTPTransport.helo(SMTPTransport.java:1663) ~[?:?]",
"at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:728) ~[?:?]",
"at javax.mail.Service.connect(Service.java:342) ~[?:?]",
"at org.elasticsearch.xpack.watcher.notification.email.Account.lambda$executeConnect$2(Account.java:164) ~[?:?]",
"at java.security.AccessController.doPrivileged(AccessController.java:554) ~[?:?]",
"at org.elasticsearch.xpack.watcher.notification.email.Account.executeConnect(Account.java:163) ~[?:?]",
"at org.elasticsearch.xpack.watcher.notification.email.Account.send(Account.java:122) ~[?:?]",
"at org.elasticsearch.xpack.watcher.notification.email.EmailService.send(EmailService.java:169) ~[?:?]",
"... 12 more"] }

What is even more incredible is that I used the same configuration and used “alert and actions” to create an email connetor. Surprisingly, it succeeded.

Any idea?

Thank you very much

leandrojmp · November 20, 2021, 6:55am

Are you sure this configuration is correct?

xpack.notification.email.account.exchange_account.smtp.host: "***@***"

smtp.host should be the host of your exchange server.

It seems that this is misconfigured, it is in the format user@domain, I think this would also explain the error you are getting:

"Caused by: javax.mail.MessagingException: 501 5.5.4 Invalid domain name"

George_Lee · November 20, 2021, 7:00am

Thank you for your reply. For safety reasons, I deliberately dealt with it this way. As you said, I use a format like user@domain. This is why email connector in alert and action can be successful。

George_Lee · November 20, 2021, 7:10am

This is a screenshot of my successful test using email connector。

George_Lee · November 20, 2021, 7:14am

However, I made an error in the watcher test, this is a screenshot of the error

leandrojmp · November 20, 2021, 2:29pm

Does the Kibana email connector works? What did you put in the Host field in the email connector configuration?

The smtp.host settings needs to be a host name.

George_Lee · November 22, 2021, 3:19am

Thank you very much @leandrojmp. I have found the problem, javax.mail handles the host a bit special, I processed the configuration in /etc/hosts in the container, and it is already working.

system · December 20, 2021, 3:19am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.