Not able to trigger email from connector

desai.aditya12 · July 14, 2022, 4:38pm

I am configuring a connector to send emails when server is down. for that i have done gmail configuration in elasticsearch.yml
xpack.notification.email.account:
gmail_account:
profile: gmail
smtp:
auth: true
starttls.enable: true
host: smtp.gmail.com
port: 587
user: test.test@somemail.com
Also executed elasticsearch-keystore add xpack.notification.email.account.gmail_account.smtp.secure_password
to set password.
The same username and password i have mentioned in connector.
The same password i have mentioned in connector config.
Despite of providing valid creds I am getting the issue.

Patrick_Mueller · July 14, 2022, 5:16pm

The xpack.notification.email.* settings are for Watcher and specified in the elasticsearch configuration (and keystore), but are not used for Kibana alerting, which it looks like you're using.

There are some notes on the gmail usage here: Email connector and action | Kibana Guide [8.11] | Elastic

The other potential problem is that you need to re-enter the userid / password each time you edit the connector. If you make any change to the connector, the credentials need to be re-entered, but I think recent releases of Kibana make that fairly clear (and perhaps prevent saving without them), compared to older releases where it may not have been.

desai.aditya12 · July 14, 2022, 7:08pm

In the link provided —> sending email from gmail there is configuration mentioned. Should It be added to elasticsearch.yml or kibana.yml?

My aim is to trigger mail when the kibana server goes down.
Is my approach proper or is there any other alternative?

hendry.lim · July 15, 2022, 1:35am

You will have to use Elasticsearch Watcher if you intend to send an alert if Kibana is down.

desai.aditya12 · July 15, 2022, 4:22am

Patrick_Mueller
Where shall i add(kibana.yml or elasticsearch.yml) email connector configuration?

desai.aditya12 · July 15, 2022, 6:46am

@hendry.lim
I have configured a watcher and it's getting triggered but still I am getting error while sending email.
Please find configuration in elasticsearch.yml
watcherconfig

{
  "trigger": {
    "schedule": {
      "interval": "10s"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "heartbeat-*"
        ],
        "rest_total_hits_as_int": true,
        "body": {
          "query": {
            "bool": {
              "must": {
                "match": {
                  "monitor.status": "down"
                }
              },
              "filter": {
                "range": {
                  "@timestamp": {
                    "from": "now-50s"
                  }
                }
              }
            }
          },
          "aggregations": {
            "by_monitors": {
              "terms": {
                "field": "monitor.id",
                "size": 10,
                "min_doc_count": 1
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gt": 0
      }
    }
  },
  "actions": {
    "email_admin": {
      "email": {
        "profile": "standard",
        "to": [
          "test@testmail.com"
        ],
        "subject": "Server down"
      }
    }
  }
}

[2022-07-15T11:41:01,474][ERROR][o.e.x.w.a.e.ExecutableEmailAction] [node-1] failed to execute action [634b140e-8d33-480d-988a-a06ad4139a1f/email_admin]
javax.mail.MessagingException: failed to send email with subject [Monitor is DOWN: monitorname] via account [gmail_account]
        at org.elasticsearch.xpack.watcher.notification.email.EmailService.send(EmailService.java:278) ~[?:?]
        at org.elasticsearch.xpack.watcher.notification.email.EmailService.send(EmailService.java:238) ~[?:?]
        at org.elasticsearch.xpack.watcher.actions.email.ExecutableEmailAction.execute(ExecutableEmailAction.java:95) ~[?:?]
        at org.elasticsearch.xpack.core.watcher.actions.ActionWrapper.execute(ActionWrapper.java:190) [x-pack-core-8.3.2.jar:?]
        at org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:563) [x-pack-watcher-8.3.2.jar:8.3.2]
        at org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:342) [x-pack-watcher-8.3.2.jar:8.3.2]
        at org.elasticsearch.xpack.watcher.execution.ExecutionService.lambda$executeAsync$6(ExecutionService.java:440) [x-pack-watcher-8.3.2.jar:8.3.2]
        at org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:669) [x-pack-watcher-8.3.2.jar:8.3.2]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:710) [elasticsearch-8.3.2.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
        at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: javax.mail.AuthenticationFailedException: 535-5.7.8 Username and Password not accepted. Learn more at
535 5.7.8  https://support.google.com/mail/?p=BadCredentials t21-20020a05620a451500b006af37b005bdsm3136617qkp.128 - gsmtp

I have set actual gmail id and password.

hendry.lim · July 15, 2022, 7:01am

As the error mentioned:
Caused by: javax.mail.AuthenticationFailedException: 535-5.7.8 Username and Password not accepted.

If you have 2FA configured in Gmail, you may need to generate application code/key instead of using username/password to authenticate.

desai.aditya12 · July 15, 2022, 7:14am

@hendry.lim
I can see than in my gmail setting i have set 2-step verification off.
Is there any setting to override it in kibana/elasticsearch side?

With respect to the same I am getting below error
Caused by: javax.mail.AuthenticationFailedException: 535-5.7.8 Username and Password not accepted. Learn more at
535 5.7.8 Check Gmail through other email platforms - Gmail Help a9-20020a05620a124900b006af08c26774sm3233505qkl.47 - gsmtp

Also 2FA is not enabled.
How can I make this work?

hendry.lim · July 15, 2022, 7:25am

You may need to enable this setting in your Google account:
smtp error Error: Invalid login: 535-5.7. 8 Username and Password not accepted. It typically indicates that you did not enable Less Secure Apps on the inbox side of your email address.

desai.aditya12 · July 15, 2022, 7:54am

@hendry.lim
Sign in with App Passwords - Google Account Help g14-20020a05620a40ce00b006af3bc9c6bbsm4128401qko.52 - gsmtp

I am not getting any option in gmail side to enable Less Secure Apps. Since that feature is not available

desai.aditya12 · July 15, 2022, 7:56am

Isn't it as simple as creating a watcher and triggering mail. I am only stuck because of
Caused by: javax.mail.AuthenticationFailedException: 535-5.7.8 Username and Password not accepted. Learn more at
535 5.7.8 Check Gmail through other email platforms - Gmail Help l67-20020a37bb46000000b006b53fe19c41sm3368597qkf.14 - gsmtp

desai.aditya12 · July 18, 2022, 6:32am

@hendry.lim
for some reason still I am getting the below error
Caused by: javax.mail.AuthenticationFailedException: 535-5.7.8 Username and Password not accepted. Learn more at
535 5.7.8 Check Gmail through other email platforms - Gmail Help d24-20020a05620a167800b006b5cb5d2fa0sm2663987qko.1 - gsmtp

I have disabled 2FA in my gmail acc. also there is not option to enable less secure apps

desai.aditya12 · July 18, 2022, 6:35am

@hendry.lim
Is there any other way to configure yahoo mail id in elasticsearch?

hendry.lim · July 18, 2022, 7:10am

I would suggest to create an app password with Gmail.

You can configure multiple email accounts under xpack.notification.email.*.

desai.aditya12 · July 18, 2022, 10:26am

@hendry.lim
It worked for gmail but for outlook it's failing,
getting below error:

Caused by: javax.mail.AuthenticationFailedException: 535 5.7.139 Authentication unsuccessful, the request did not meet the criteria to be authenticated successfully. Contact your administrator. [CH2PR03CA0027.namprd03.prod.outlook.com]

configuration:

xpack.notification.email.account:
    outlook_account:
        profile: outlook
        smtp:
            auth: true
            starttls.enable: true
            host: smtp-mail.outlook.com
            port: 587
            user: test@testmail.com

Any solution for this.
I have set proper password too

hendry.lim · July 18, 2022, 11:26pm

All the errors you were encountering had nothing to do with Elasticsearch. I would suggest you to resolve the errors by testing the access with an email client. Once that's working, use the same configuration in Elasticsearch.

desai.aditya12 · July 28, 2022, 9:35am

Hi @hendry.lim
I am trying to use elastalert with heartbeat and elasticsearch, to trigger email when site goes down.
Please find below configuration file for example_frequency.yaml

# Alert when the rate of events exceeds a threshold

# (Optional)
# Elasticsearch host
# es_host: elasticsearch.example.com

# (Optional)
# Elasticsearch port
# es_port: 14900

# (OptionaL) Connect with SSL to Elasticsearch
#use_ssl: True

# (Optional) basic-auth username and password for Elasticsearch
#es_username: someusername
#es_password: somepassword

# (Required)
# Rule name, must be unique
name: Heartbeat trigger

# (Required)
# Type of alert.
# the frequency rule type alerts when num_events events occur with timeframe time
type: frequency

# (Required)
# Index to search, wildcard supported
index: heartbeat-*

# (Required, frequency specific)
# Alert when this many documents matching the query occur within a timeframe
num_events: 1

# (Required, frequency specific)
# num_events must occur within this amount of time to trigger an alert
timeframe:
  seconds: 4

# (Required)
# A list of Elasticsearch filters used for find events
# These filters are joined with AND and nested in a filtered query
# For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html
filter:
- term:
    monitor.status: "up"

# (Required)
# The alert is use when a match is found
alert:
- "email"

# (required, email specific)
# a list of email addresses to send alerts to
email:
- "some mail id"

While running python -m elastalert.elastalert --verbose --rule example_frequency.yaml
I am getting below error:

Is there any configuration which i am missing?

richcollier · July 28, 2022, 10:13am

Elastalert is not an Elastic product, it is an open-source project started by Yelp. Please seek help for that product elsewhere or use Kibana Alerting (which we do make). We'd be happy to help you with Kibana Alerting.

system · August 25, 2022, 10:14am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.