Logstash error while using email plugin

Elastic Stack Logstash
1

I am trying to simulate an environment wherein if the temperature exceeds 40℉, an email is sent to the user. I am running logstash on my local.

Here is the config file.

weather_data.conf

input {
  http_poller {
    urls => {
        weather => {
            url => "http://api.openweathermap.org/data/2.5/weather?id=5490223&appid="MY_APP_ID"&units=metric"
            headers => {
              Accept => "application/json"
            }
        }
    }
    schedule => { cron => "* * * * * *" }
    codec => json
  }
}
filter {
  mutate {
    remove_field => ["@version" ,"command" ,"host" ,"cod" ,"id" ,"base" ,"coord" ,"sys" ,"dt"]
  }
  ruby{
	code => ' if Time.now.to_i % 120 == 0 ; event.set("[main][temp]", 15 + event.get("[main][temp]")); end'
  }
  split { field => "weather" }
}
output {
   if [main][temp] >= 40 {
	email {
	  to => 'john@gmail.com'
	  subject => 'Alert - Temperature exceeded threshold'
	  body => "Exceeded temperature"
	}
  }
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "weather"
  }
  stdout {
    codec => rubydebug
  }
}

Exactly two minutes after running the config file, I get an error like this,

[2018-08-02T20:22:22,004][INFO ][logstash.inputs.http_poller] Registering http_poller Input {:type=>nil, :schedule=>{"cron"=>"* * * * * *"}, :timeout=>nil}
[2018-08-02T20:22:22,082][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0xca32fb8 run>"}
[2018-08-02T20:22:22,238][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-08-02T20:22:22,898][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2018-08-02T20:24:02,700][ERROR][logstash.outputs.email   ] Something happen while delivering an email {:exception=>#<Errno::ECONNREFUSED: Connection refused - connect(2) for "localhost" port 25>}

What wrong am I doing? How can I make sure the email is sent? Do I have to include the logstash port?

Can someone please help me out here? I would really appreciate if you could. Thanks in advance.

2

How are you planning to send the email? Do you SMTP access or will it be a mailbox?

You might need to specify the address or domain and the auth setting if needed to send the email.

Here is another config that you could look at:

3

Also, are you running anything like postfix or sendmail locally? Try doing telnet localhost 25.

4

Hi NerdSec,

Not really sure what SMTP access is. I haven't actually set anything. For testing purpose, I only want to send an email to my personal email ID (I presume this means mailbox). Also, logstash is running on localhost. How should I go about doing this?

5

I tried running telnet localhost 25

I get the following error,

PS C:\Windows\System32\WindowsPowerShell\v1.0> telnet localhost 25
Connecting To localhost...Could not open connection to the host, on port 25: Connect failed

Could you please tell me how I can fix this?

6

Ok. So you don't have access to an SMTP or a mailbox locally.

Why don't you send an email via Gmail itself?

Try doing the following:
telnet smtp.gmail.com 587

7

If you are able to telnet then try using the following config:

email {
  to => "xyz@gmail.com"
  body => "Message: %{message}"
  address => "smtp.gmail.com"
  port => 587
  username => "xyz@gmail.com"
  password => "mypass"
  use_tls => true
}
8

Thanks! I did as you said.

I got an output like this,

220 smtp.gmail.com ESMTP s14-v6sm8644374pfj.105 - gsmtp

Also, As per my logstash configuration I am sending an email through gmail right?

9

Here is my updated logstash config

input {
  http_poller {
    urls => {
        weather => {
            url => "http://api.openweathermap.org/data/2.5/weather?id=5490223&appid="MY_APP_ID"&units=metric"
            headers => {
              Accept => "application/json"
            }
        }
    }
    schedule => { cron => "* * * * * *" }
    codec => json
  }
}
filter {
  mutate {
    remove_field => ["@version" ,"command" ,"host" ,"cod" ,"id" ,"base" ,"coord" ,"sys" ,"dt"]
  }
  ruby{
	code => ' if Time.now.to_i % 15 == 0 ; event.set("[main][temp]", 15 + event.get("[main][temp]")); end'
  }
  split { field => "weather" }
}
output {
  email {
    to => "myusername@gmail.com"
    body => "Message: %{message}"
    address => "smtp.gmail.com"
    port => 587
    username => "myusername@gmail.com"
    password => "mypassword"
    use_tls => true
  }
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "weather"
  }
  stdout {
    codec => rubydebug
  }
}

When i try running the config file, I get an error like this,

[2018-08-02T23:15:58,498][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2018-08-02T23:16:02,653][ERROR][logstash.outputs.email   ] Something happen while delivering an email {:exception=>#<Net::SMTPAuthenticationError: 534-5.7.14 <https://accounts.google.com/signin/continue?sarp=1&scc=1&plt=AKgnsbsQ
>}
[2018-08-02T23:16:03,596][ERROR][logstash.outputs.email   ] Something happen while delivering an email {:exception=>#<Net::SMTPAuthenticationError: 534-5.7.14 <https://accounts.google.com/signin/continue?sarp=1&scc=1&plt=AKgnsbvY
>}
10

Are you using 2FA with Gmail? If so, you might have to use App passwords.

https://support.google.com/accounts/answer/185833

11

I am not using 2FA. 2FA is not available for my account.

Capture
Capture.PNG933×557 48.4 KB

12

Then, are you sure the password is correct? It seems to be an authentication error.

13

Yes. Password is absolutely right. I even successfully logged in through using chrome browser.

14

I have successfully tested this on my setup and the following config works perfectly fine:

email {
      to => "nerdsec@gmail.com"
      body => "Message: %{message}"
      address => "smtp.gmail.com"
      port => 587
      username => "nerdsec@gmail.com"
      password => "lasdasd"
      use_tls => true
}

I am using 2FA so, have used an App password for authentication.

15

You need to enable "Less secure apps" (sic) in your Gmail account if you want to connect to it via SMTP without a dedicated app password: https://support.google.com/accounts/answer/6010255?hl=en

16

Thanksa lot! It works now! :slight_smile:

17

I had to change my google settings to allow less secure apps! After making the settings, it worked! Thanks a lot again NerdSec! :slight_smile: