Trigger email event(output) on file input which having csv filter

while using logstash i stuck for sending email on event when i found list of specific expression in input file
which is filter by csv plugin

Wrap your email output in a conditional that looks at the input event to decide what to do.

https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html

i read input from file where logs is created like this

2018-04-04T09:15:37+0000,vagrant-ubuntu-trusty-64,vagrant,1000,1459,/dev/pts/0,"/home/vagrant","/bin/su","su USERNAME"
2018-04-04T09:15:39+0000,vagrant-ubuntu-trusty-64,USERNAME,1003,1459,/dev/pts/0,"/home/vagrant","/usr/bin/groups","groups"
2018-04-04T09:15:39+0000,vagrant-ubuntu-trusty-64,USERNAME,1003,1459,/dev/pts/0,"/home/vagrant","/usr/bin/lesspipe","lesspipe"
2018-04-04T09:15:39+0000,vagrant-ubuntu-trusty-64,USERNAME,1003,1459,/dev/pts/0,"/home/vagrant","/usr/bin/basename","basename /usr/bin/lesspipe"
2018-04-04T09:15:39+0000,vagrant-ubuntu-trusty-64,USERNAME,1003,1459,/dev/pts/0,"/home/vagrant","/usr/bin/dirname","dirname /usr/bin/lesspipe"
2018-04-04T09:15:39+0000,vagrant-ubuntu-trusty-64,USERNAME,1003,1459,/dev/pts/0,"/home/vagrant","/usr/bin/dircolors","dircolors -b"
2018-04-04T09:16:01+0000,vagrant-ubuntu-trusty-64,USERNAME,1003,1459,/dev/pts/0,"/home/USERNAME","/bin/rm","rm USERNAME_f"
2018-04-04T09:16:05+0000,vagrant-ubuntu-trusty-64,USERNAME,1003,1459,/dev/pts/0,"/home/USERNAME","/bin/rm","rm USERNAME_file"

i use csv filter

csv{
separator => ","
columns => ["DateTime" ,"hostname" ,"username" ,"uid","sid","tty","cwd","filename","cmdline"]

on output

there is one plug-in go for elasticsearch

and other plugin of email

if "/bin/rm" in [columns] {
email{
address => "smtp.gmail.com"
domain => "smtp.gmail.com"
port => 25
username => ""
password => ""
from => ""
subject => "Error status"
to => ""
body => "Here is the event line that occured: %{@message}"
htmlbody => "
}
}

but mail is not trigger

might be condition is wrongly stated
of mail setting is wrong

but i configure smtp mail in this linux machine and able to sent mail via terminal
but not via logstash output

Have you looked in the Logstash log for clues?

If your ISP allows you to connect to smtp.gmail.com:25 you'll only be able to send email to Gmail-hosted addresses. If you want to use Gmail as an SMTP relay you need to authenticate and use SSL or TLS.

I suggest you configure a local MTA and have Logstash connect to it instead. Then you can configure SMTP in a single play and have any number of local daemons send email.

is my this condition is correct
the columns i want to check is filename as mention in csv filter
AND
i saw logstash-plain.log file in /opt/logstash/logs/logstash-plain.log

here is logs coming repetitively

[2018-04-04T09:03:09,644][WARN ][logstash.outputs.Elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"snoopy", :_type=>"doc", :_routing=>nil}, #LogStash::Event:0x202f72e0], :response=>{"index"=>{"_index"=>"firstindex", "_type"=>"doc", "id"=>"ojLlj2IBGlko91sbQqSv", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [DateTime]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: " s/^IF_OFFLOAD//; s/=.*//; s...""}}}}}

yes it also show this too

[2018-04-04T10:33:14,655][ERROR][logstash.outputs.email ] Something happen while delivering an email {:exception=>#<Net::SMTPAuthenticationError: 530 5.7.0 Must issue a STARTTLS command first. b6sm9873083pfm.160 - gsmtp

is my this condition is correct

No, since you don't have a columns field. It looks like "/bin/rm" ends up in the filename field so you can do this instead:

if [filename] == "/bin/rm" {

The ES error in the log is because you for some reason have ended up with garbage in the DateTime field. I don't know what "s/^IF_OFFLOAD//; s/=.*/" comes from.

The email output error indicates that you need to enable TLS in the plugin.

thanks i corrected my condition and the that DateTime issue is might be due to some mutate filter option put it

Thanks for help

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.