Logstash: Comparison of two variables

Elastic Stack Logstash
1

I wrote a logstash config like that, with a lot of if [message]. In this example code below, Im filtering logs and have the problem, that a grok named (%{EMAILADDRESS:sender}) (the opposide of (%{EMAILADDRESS:recipient}) ) is written in my output, which I don't use in this if block . I don't get any grokparse errors and I'm 100% in this if block. For debugging reasons I named to something else and the result was how I assumed.
So my (nasty) idea for a workaround is like (%{EMAILADDRESS:recipient}) == (%{EMAILADDRESS:sender}) then remove_field => "sender".

What's the syntax for comparing these both pattern?

input { 
        file {
                id => "main"
                path => "/usr/share/logstash/mainlog.log"
                sincedb_path => "/dev/null"
                start_position => "beginning"
        }
        stdin { } 
}
filter { 

[…]
            if [flags] == "**" {
			if [message] =~ "SMTP error from remote mail server after RCPT TO" {
				grok {
					"match" => { "message" => " (%{EMAILADDRESS:recipient})" }
				}
				mutate {
					update => { "exim_msg_state" => "error from remote" }
					lowercase => [ "recipient" ]
				}
			}
                }

[…]

}

output {
        elasticsearch { 
                hosts => ["elasticsearch:9200"]
        }
        stdout { codec => rubydebug }
}
2

Hello,

It is not really clear what you are trying to do and what is your issue.

You can't have conditionals inside a grok pattern, you can just compare fields by using if [recipient] == [sender]

Can you provide more context and some sample documents of what you are trying to achieve and what is the result?

1 Like
3

sure I can.

this example below was just for example in in "pseudocode". I tested it in this way.
My idea was to compare, if in the (%{EMAILADDRESS:recipient}) is equal to (%{EMAILADDRESS:sender}). If yes, then remove field sender.
So I have to check if the contenct is equal and put remove_field in the mutate block.

Edit: thank you for this solution it works now,

4

Can you show that part of .conf, how have you implemented?

5

yes.

input { 
        file {
                id => "main"
                path => "/usr/share/logstash/mainlog.log"
                sincedb_path => "/dev/null"
                start_position => "beginning"
        }
        stdin { } 
}
filter { 

[…]
            if [flags] == "**" {
			if [message] =~ "SMTP error from remote mail server after RCPT TO" {
				grok {
					"match" => { "message" => " (%{EMAILADDRESS:recipient})" }
				}
				mutate {
					update => { "exim_msg_state" => "error from remote" }
					lowercase => [ "recipient" ]
				}
                                if [recipient] == [sender] {
                                   # dirty workaround
                                    mutate {
                                      remove_field  => "sender"
                                    }
                          }
			}
                }

[…]

}

output {
        elasticsearch { 
                hosts => ["elasticsearch:9200"]
        }
        stdout { codec => rubydebug }
}

edit: I know the yml syntax doesnt looks nice. I hope you can read and understand it.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.