Logstash convert Nil values into a number

Elastic Stack Logstash
1

Hello everyone,

I'm trying to find the fields that are empty and change them for a value.

I want to do something like the next expression, but Logstash can not find the nil value. Does anyone have a suggestion for finding and changing empty fields?

filter{
if [field] == nil {
mutate {
replace => { "field" => "99999" }
}
}
}

Thank you

2

Empty meaning "contains an empty string" or "doesn't exist at all" or something else?

3

The original log is a semi-colon separated field and, for instance when you have 4 empty fields it's showed as ";;;" which is in fact are 4 empty strings but they are supposed to be 4 ints. Some of the field's values are "regular" int values (1,2,3,4), some have 0, but I want to replace the ones that don't have anything with 0 (zero).

4

In that case use if [field] == "" {.

5

I try to use that expression but didn't change the values.

6

Show an example document that doesn't end up as expected. Use a stdout { codec => rubydebug } output to dump the raw event.

7

An example of the output:
"
"field" => nil,
"@version" => "1",
"field2" => 0,
"field3" => 0,
"field4" => 0,
"field5" => nil,
"field8" => 0,
"message" => "190123001;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;;;;;;;;;;;;;;;;;;;;;",
"ind_filedate" => "20161118",
"field5" => nil,
"field6" => nil,
"field7" => nil,
"@timestamp" => 2016-11-18T00:00:00.000Z"

8

Okay. I don't undertand how the nil ends up there, but try if [field5] {. If that doesn't work you'll have to use a ruby filter.

9

Didn't work also. I don't know ruby code this, can you post an example please?

10

Something like this should work:

if event.get('field').nil?
  event.set('field', 9999)
end

Lots of ruby filter examples have been posted here in the past.

1 Like
11

That ruby code work perfectly. Thank you Magnus :slight_smile:

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.