How to enable authentication with logstash email output

learningelastic · November 8, 2022, 8:28pm

I want to learn how to use logstash with mailtrap smtp for development purposes. I installed logstash then ran this command:

/usr/share/logstash/bin/logstash -e 'input { stdin { } } output { email {
		to => "user@example.com"
		from => "user@example.com"
		subject => "Alert - %{title}"
		body => "content here"
		authentication => "plain"
		domain => "smtp.mailtrap.io:2525"
		username => "20ff3475e0c350"
		password => "594980b5a1be46"
    }
}'

Once logstash is up and running, I type something and press enter. This causes the error below:

[ERROR] 2022-11-08 19:34:59.861 [[main]>worker1] email - Something happen while delivering an email {:exception=>#<Net::SMTPAuthenticationError: 503 5.5.1 Error: authentication not enabled

How do I enable authentication? Or what am I doing wrong? I also can't find any documentation on what are acceptable values for the authentication property.

tatdat · November 9, 2022, 5:59am

You have the incorrect configuration:

 {
		to => "user@example.com"
		from => "user@example.com"
		subject => "Alert - %{title}"
		body => "content here"
		authentication => "plain"
		domain => "smtp.mailtrap.io" #Pplease enter domain only, not include port
        port =>2525 # port used to communicate with the mail server
		username => "20ff3475e0c350"
		password => "594980b5a1be46"
    }

You can view more detail about email configuration in here
Email output plugin | Logstash Reference [8.5] | Elastic

learningelastic · November 9, 2022, 6:17am

tatdat:

{
		to => "user@example.com"
		from => "user@example.com"
		subject => "Alert - %{title}"
		body => "content here"
		authentication => "plain"
		domain => "smtp.mailtrap.io" #Pplease enter domain only, not include port
        port =>2525 # port used to communicate with the mail server
		username => "20ff3475e0c350"
		password => "594980b5a1be46"
    }

Thanks for suggestion. I just tried your configuration. Now the error I get is:

[ERROR] 2022-11-09 06:16:40.507 [[main]>worker0] email - Something happen while delivering an email {:exception=>#<Errno::ECONNREFUSED: Connection refused - connect(2) for "localhost" port 2525>}

The credentials i shared in my original post are real and you can test with the same details to confirm the same errors.

tatdat · November 9, 2022, 6:21am

Which is the logstash version you are using?

learningelastic · November 9, 2022, 2:04pm

I tried on logstash 8.4.1 and logstash 8.5.0 and they both gave the same errors.

Did you actually get a success response on your end when you ran the logstash command? If so, can you post the response? And let me know which logstash version you used?

Thanks

tatdat · November 9, 2022, 2:12pm

From your logstash server, try to test connect to mail server with command

telnet smtp.mailtrap.io 2525

Is that working?

btw, try to add this args in the output email config

{
		.......
		domain => "smtp.mailtrap.io"
        address=> "smtp.mailtrap.io"
        .......
    }

learningelastic · November 9, 2022, 2:17pm

Yes, I got this response:

telnet smtp.mailtrap.io 2525
Trying 3.219.2.182...
Connected to mailsend-smtp-classic-f3a4534c019a3e96.elb.us-east-1.amazonaws.com.
Escape character is '^]'.
220 smtp.mailtrap.io ESMTP ready

I also have a web application that has no trouble connecting and submitting emails to smtp.mailtrap.io on port 2525.

I also tried many combinations of address, domain, port and they all gave the ECONNREFUSED error. The only combination that gave a different error was the configuration i posted in my original question -- with the error being authentication not enabled.

learningelastic · November 9, 2022, 2:26pm

I just tried this

/usr/share/logstash/bin/logstash -e 'input { stdin { } } output { email {
    to => "user@example.com"
    from => "user@example.com"
    subject => "Alert - %{title}"
    body => "content here"
    authentication => "plain"
    address => "smtp.mailtrap.io"
    domain => "smtp.mailtrap.io"
    port =>2525
    username => "20ff3475e0c350"
    password => "594980b5a1be46"
    }
}'

The system just hangs like this:

Using bundled JDK: /usr/share/logstash/jdk
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2022-11-09 14:25:02.474 [main] runner - NOTICE: Running Logstash as superuser is not recommended and won't be allowed in the future. Set 'allow_superuser' to 'false' to avoid startup errors in future releases.
[INFO ] 2022-11-09 14:25:02.491 [main] runner - Starting Logstash {"logstash.version"=>"8.5.0", "jruby.version"=>"jruby 9.3.8.0 (2.6.8) 2022-09-13 98d69c9461 OpenJDK 64-Bit Server VM 17.0.4+8 on 17.0.4+8 +indy +jit [x86_64-linux]"}
[INFO ] 2022-11-09 14:25:02.495 [main] runner - JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[WARN ] 2022-11-09 14:25:02.868 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2022-11-09 14:25:04.561 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[INFO ] 2022-11-09 14:25:05.240 [Converge PipelineAction::Create<main>] Reflections - Reflections took 178 ms to scan 1 urls, producing 125 keys and 438 values
[INFO ] 2022-11-09 14:25:05.874 [Converge PipelineAction::Create<main>] javapipeline - Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[INFO ] 2022-11-09 14:25:06.863 [[main]-pipeline-manager] javapipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>250, "pipeline.sources"=>["config string"], :thread=>"#<Thread:0x27564684 run>"}
[INFO ] 2022-11-09 14:25:07.513 [[main]-pipeline-manager] javapipeline - Pipeline Java execution initialization time {"seconds"=>0.65}
[INFO ] 2022-11-09 14:25:07.587 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"}
The stdin plugin is now waiting for input:
[INFO ] 2022-11-09 14:25:07.658 [Agent thread] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
Typing another test then press enter

tatdat · November 9, 2022, 2:36pm

It seems like the error disappeared.
Add --debug on the Logstash run command , After logstash run success, enter sometext send see output.

/usr/share/logstash/bin/logstash --debug -e 'input { stdin { } } output { email {
    to => "user@example.com"
    from => "user@example.com"
    subject => "Alert - %{title}"
    body => "content here"
    authentication => "plain"
    address => "smtp.mailtrap.io"
    domain => "smtp.mailtrap.io"
    port =>2525
    username => "20ff3475e0c350"
    password => "594980b5a1be46"
    }
}'

When you see the log

[INFO ] 2022-11-09 14:25:07.587 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"}
The stdin plugin is now waiting for input:
[INFO ] 2022-11-09 14:25:07.658 [Agent thread] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
Typing another test then press enter

Enter some text to test

learningelastic · November 9, 2022, 2:40pm

Oh wait!! it actually worked! The system wasn't actually hanging. It's just natural behaviour to not report anything when success. So to confirm, your suggesion with this command yielded success:

/usr/share/logstash/bin/logstash -e 'input { stdin { } } output { email {
    to => "user@example.com"
    from => "user@example.com"
    subject => "Alert - %{title}"
    body => "content here"
    authentication => "plain"
    address => "smtp.mailtrap.io"
    domain => "smtp.mailtrap.io"
    port =>2525
    username => "20ff3475e0c350"
    password => "594980b5a1be46"
    }
}'

Thank you so much !!! I will create a youtube video and share with the internet

tatdat · November 9, 2022, 2:50pm

Glad to help you. I'm looking forward to your video, hehe
One more thing, if user/pass is TRUE, please remove it or change to *******

learningelastic · November 10, 2022, 5:28pm

Thanks, here's the video I made thanks to your assistance!

I used your mailtrap solution at 14:50 of the video.

system · December 8, 2022, 5:29pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.