Email not sent when inviting new users "block in start_queue_manager"

I try to invite users using app search/Users Invite user using the standard authentification mode. The invitation is shown as pending, and I tried to resend mail many times. Email is never received, I checked of course spam, and different destination addresses.
I have a self host config with a working elasticsearch and enterpise-search installation on centos 8
What I set in my /config/enterprise-search.yml to enable email is:

    ent_search.auth.default.source: standard
     email.account:
       enabled: true
       smtp:
         auth: login 
         starttls.enable: true
         host: https://smtp-mail.outlook.com
         port: 587
         user:xxxx@xxxxx.xxx
         password: xxxxxxx
       email_defaults:
         from: xxx@xxxxxx.xxx

I also try with a gmail account, and also with a smtp server related to documentation.
I set the logs in debug mode and I can see the email in it, with at the end a message "block in start_queue_manager"

Do you have any ideas ? Thanks a lot !

debug log :

[2021-03-21T19:54:53.817+00:00][788962][2306][app-server][INFO]: [ActiveJob] [Work::TogoMailer] [66047d10-e0cb-4b4e-9df4-8f84180db4d0] 
Sent mail to xxxx@xxxxx.xxx (104.3ms)
[2021-03-21T19:54:53.817+00:00][788962][2306][app-server][DEBUG]: [ActiveJob] [Work::TogoMailer] [66047d10-e0cb-4b4e-9df4-8f84180db4d0] Date: Sun, 21 Mar 2021 19:54:53 +0000
From: xxxx@xxxxx.xxx
To: xxxx@xxxxx.xxx
Message-ID: <63d5ec0d72441f839ada3c4bcec240bb220da99a319c23ed4dce6768a894464b-xxxx@xxxxx.xxx
Subject: enterprise_search has invited you to join App Search!
Mime-Version: 1.0
Content-Type: multipart/alternative;
 boundary="--==_mimepart_6057a48da81bc_c09e2902-4ce";
 charset=UTF-8
Content-Transfer-Encoding: 7bit

(I removed the html email content here)

[2021-03-21T19:54:53.819+00:00][788962][2306][active_job][INFO]: [ActiveJob] [Work::TogoMailer] [66047d10-e0cb-4b4e-9df4-8f84180db4d0] Completed Work::TogoMailer job (66047d10-e0cb-4b4e-9df4-8f84180db4d0) on `mailer`
[2021-03-21T19:54:53.821+00:00][788962][2306][active_job][INFO]: [ActiveJob] [Work::TogoMailer] [66047d10-e0cb-4b4e-9df4-8f84180db4d0] Performed Work::TogoMailer from EsqueuesMe(mailer) in 227.09ms
[2021-03-21T19:54:53.835+00:00][788962][2336][es][DEBUG]: {
  "request": {
    "url": "http://127.0.0.1:9200/.ent-search-actastic-crawler_crawl_requests_v3/_search?request_cache=true",
    "method": "get",
    "headers": {
      "Authorization": "[FILTERED]",
      "Content-Type": "application/json",
      "User-Agent": "Faraday v0.17.3"
    },
    "params": null,
    "body": "{\"query\":{\"bool\":{\"filter\":[{\"terms\":{\"status\":[\"pending\"]}}]}},\"sort\":[{\"created_at\":\"desc\"},\"_doc\"],\"size\":100,\"from\":0,\"seq_no_primary_term\":true}"
  },
  "response": {
    "status": 200,
    "headers": {
      "content-type": "application/json; charset=UTF-8"
    },
    "body": "{\"took\":1,\"timed_out\":false,\"_shards\":{\"total\":1,\"successful\":1,\"skipped\":0,\"failed\":0},\"hits\":{\"total\":{\"value\":0,\"relation\":\"eq\"},\"max_score\":null,\"hits\":[]}}"
  },
  "duration": 14.8,
  "stack": [
    "lib/actastic/schema.class:22:in `search'",
    "lib/actastic/relation.class:488:in `search'",
    "lib/actastic/relation.class:216:in `find_each'",
    "lib/actastic/relation.class:287:in `to_a'",
    "lib/actastic/relation.class:287:in `load'",
    "lib/actastic/relation.class:282:in `to_a'",
    "lib/actastic/relation.class:303:in `empty?'",
    "shared_togo/lib/shared_togo/crawler/queue_manager.class:155:in `perform_queue_polling'",
    "shared_togo/lib/shared_togo/crawler/queue_manager.class:95:in `block in start!'",
    "shared_togo/lib/shared_togo/crawler/queue_manager.class:86:in `loop'",
    "shared_togo/lib/shared_togo/crawler/queue_manager.class:86:in `start!'",
    "shared_togo/lib/shared_togo/crawler.class:16:in `block in start_queue_manager!'"
  ]
}

Hi Patrice, sorry for the late reply.

It seems that you tried all the obvious debugging steps, so we'll need to dig deeper.

To exclude problems with Outlook and Gmail servers, I think we could try using a local SMTP server.

You said that you tried using an SMTP server related to documentation. Are talking about the first config on this page? Inviting users: SMTP setup | Workplace Search Guide [7.12] | Elastic

Also want to confirm that you are running Enterprise from a Linux package and not from a Docker image?

Hi Vadim, thank you for your support. I am not using Docker, but run Enteprise directly on centos 8
I installed a local mail server, on the same server, and I can send and receive a noreply test mail to my personnal mail using this command :

su noreply
echo "test email4" | sendmail patrice.xxxx@xxxx.com

Then I retry, after a restart in app search | invite user, but no mail is received.
For that configuration I used as proposed by documentation :

email.account:
  enabled: true
  smtp:
    auth: plain # Can be plain, login, or CRAM-MD5.
    starttls.enable: false # Whether emails are sent encapsulated in TLS, or not.
    host: 127.0.0.1
    port: 25
    user: noreply@XXX.com #I also tried without @xxx.com
    password: xxxxxx   #the password for my noreply user
  email_defaults:
    from: noreply@xxxx.com

and here are the standards logs when I click on resend :

[2021-03-24T21:02:29.878+00:00][868197][2476][app-server][INFO]: [fc3456b4-e5d5-47cc-93fa-7b59165b5c5e] Started POST "/as/invitations/rZ4uLdkPP5KprzPXiPUqRQ/resend" for XX.XX.XX.XX at 2021-03-24 21:02:29 +0000
[2021-03-24T21:02:29.944+00:00][868197][2476][action_controller][INFO]: [fc3456b4-e5d5-47cc-93fa-7b59165b5c5e] Processing by LocoMoco::Dashboard::InvitationsController#resend as JSON
[2021-03-24T21:02:29.945+00:00][868197][2476][action_controller][INFO]: [fc3456b4-e5d5-47cc-93fa-7b59165b5c5e]   Parameters: {"code"=>"[FILTERED]"}
[2021-03-24T21:02:30.007+00:00][868197][2476][worker][INFO]: [fc3456b4-e5d5-47cc-93fa-7b59165b5c5e] [ActiveJob] Enqueueing a job into the '.ent-search-esqueues-me_queue_v1_mailer' index. {"job_type"=>"ActiveJob::QueueAdapters::EsqueuesMeAdapter::JobWrapper", "payload"=>{"args"=>[{"job_class"=>"Work::TogoMailer", "job_id"=>"8a6f3688-a3a3-484a-a400-3594b7a3f0dc", "queue_name"=>"mailer", "arguments"=>["LocoMoco::Dashboard::InvitationMailer", "invitation", "deliver_now", ["rZ4uLdkPP5KprzPXiPUqRQ"]], "locale"=>:en, "executions"=>1}]}, "status"=>"pending", "created_at"=>1616619750007, "perform_at"=>1616619750006, "attempts"=>0}
[2021-03-24T21:02:30.035+00:00][868197][2476][active_job][INFO]: [fc3456b4-e5d5-47cc-93fa-7b59165b5c5e] [ActiveJob] Enqueued Work::TogoMailer job (8a6f3688-a3a3-484a-a400-3594b7a3f0dc) on `mailer`
[2021-03-24T21:02:30.041+00:00][868197][2476][active_job][INFO]: [fc3456b4-e5d5-47cc-93fa-7b59165b5c5e] [ActiveJob] Enqueued Work::TogoMailer (Job ID: 8a6f3688-a3a3-484a-a400-3594b7a3f0dc) to EsqueuesMe(mailer) with arguments: "LocoMoco::Dashboard::InvitationMailer", "invitation", "deliver_now", ["rZ4uLdkPP5KprzPXiPUqRQ"]
[2021-03-24T21:02:30.081+00:00][868197][2476][action_controller][INFO]: [fc3456b4-e5d5-47cc-93fa-7b59165b5c5e] Completed 200 OK in 135ms (Views: 1.4ms)
[2021-03-24T21:02:31.181+00:00][868197][2300][active_job][INFO]: [ActiveJob] [Work::TogoMailer] [8a6f3688-a3a3-484a-a400-3594b7a3f0dc] Performing Work::TogoMailer from EsqueuesMe(mailer) with arguments: "LocoMoco::Dashboard::InvitationMailer", "invitation", "deliver_now", ["rZ4uLdkPP5KprzPXiPUqRQ"]
[2021-03-24T21:02:31.364+00:00][868197][2300][action_view][INFO]: [ActiveJob] [Work::TogoMailer] [8a6f3688-a3a3-484a-a400-3594b7a3f0dc]   Rendered loco_moco/app/views/loco_moco/dashboard/invitation_mailer/invitation.html.erb within layouts/frito_pie_mailer (9.5ms)
[2021-03-24T21:02:31.385+00:00][868197][2300][action_view][INFO]: [ActiveJob] [Work::TogoMailer] [8a6f3688-a3a3-484a-a400-3594b7a3f0dc]   Rendered loco_moco/app/views/loco_moco/dashboard/invitation_mailer/invitation.text.erb (8.1ms)
[2021-03-24T21:02:31.546+00:00][868197][2300][app-server][INFO]: [ActiveJob] [Work::TogoMailer] [8a6f3688-a3a3-484a-a400-3594b7a3f0dc]
Sent mail to patrice.xxxx@xxxx.com (119.4ms)
[2021-03-24T21:02:31.548+00:00][868197][2300][active_job][INFO]: [ActiveJob] [Work::TogoMailer] [8a6f3688-a3a3-484a-a400-3594b7a3f0dc] Completed Work::TogoMailer job (8a6f3688-a3a3-484a-a400-3594b7a3f0dc) on `mailer`
[2021-03-24T21:02:31.552+00:00][868197][2300][active_job][INFO]: [ActiveJob] [Work::TogoMailer] [8a6f3688-a3a3-484a-a400-3594b7a3f0dc] Performed Work::TogoMailer from EsqueuesMe(mailer) in 367.31ms

But no mail received, and I test with several mails operator, and I checked spam, others, folders, etc...

Thanks a lot !

  1. Can you confirm that the local email server is running on port 25?
  2. I compared the logs that I had and the logs that you sent and I found a small difference:
    You had:
    Performing Work::TogoMailer from EsqueuesMe(mailer)
    While I had:
    Performing Work::TogoMailer from Class(mailer)
    What version of Enterprise Search are you using? It should be under the logo in App Search UI.

Hi Vadim, yes it's running on default 25 port. The Enterprise version is 7.12.0. Perhaps I will try to ask ElasticSearch to directly manage users using native authentication, and Enterprise mapping. In that case, because I didn't see in the documentation, does elasticsearch send an email confirmation when we create a user in elasticsearch using authentication API, like enterprise-search should do ?

Hi Patrice,

If you use native authentication, emails will not be sent.

I'm a little lost in finding the root cause. Maybe we could use a workaround?

Since you are using App Search, I would not expect there to be many users that you want to invite. Would it be possible to invite all users manually? You could copy invitation links and send them to users directly.