How to write if condition inside of the logstash grok pattern?

Elastic Stack Logstash
1

My question is related to logstash grok pattern. I created below pattern that's working fine but the big problem is not string values. Sometimes; "Y" and "age" can be null so my grok pattern not create any log in elasticseach. It is not working properly. I need to tell my grok pattern :slight_smile:

if(age is null || age i empty){

updatefield["age",0]
}

but I don't know how to make it. by the way; I checked many solutions by googling but it is directly related to my problem.

input {

file {
path => ["C:/log/*.log"]
start_position => "beginning"
discover_interval => 10
stat_interval => 10
sincedb_write_interval => 10
close_older => 10
codec => multiline {
pattern => "^%{TIMESTAMP_ISO8601}|"
negate => true
what => "previous"
}
}
}

filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:formattedDate}.* X: %{DATA:X} Y: %{NUMBER:Y} Z: %{DATA:Z} age: %{NUMBER:age:int} "}
}
date {
timezone => "Europe/Istanbul"
match => ["TimeStamp", "ISO8601"]
}
json{
source => "request"
target => "parsedJson"

}   
mutate {
remove_field => [ "path","message","tags","@version"]

}

}

output {

stdout {
    codec => rubydebug
}

elasticsearch {
hosts => [ "http://localhost:9200" ]
index => "logstash-%{+YYYY.MM}"

}

}

2

My suggestion has two steps.

1- define age and Y as optional field in grok filter.
(%{NUMBER:age})?
2-check existence of age field.if it missing add with mutate.
if ![age] {
mutate { add_field => { "age" => 0} }
}

1 Like
3

When I add grok filter, "age" => 0 is adding inside json result. But the other fields disappear. I guess that I have to add if clause before grok pattern. because when I send bull to "age" it can not translate into integer. Thanks mehmet.

4

I think if statement should be after grok filter. Afaik if you define age as optional and if it is null, grok filter doesnt add age field to the event. So you can check if it is exists or not.
Sample log lines would w/and w/o age would be helpful to investigate.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.