Best way to change a list of not wanted values

Elastic Stack Logstash
1

So I have some values being fed from logstash, and they are failing input into Elastic because I have a template being used, and the values sometimes don't adhere to the tablet.

Notably, I get the values "-" and "(empty)", which don't work well in a field designated as an integer or double...

What I would like to do is designate these values as nulls within logstash, because that's how they should be treated (that's what they really are). An Elastic field designated as "integer" doesn't puke if it's given just a null value.

What is the best way to do this? Do I need to use a ruby filter, and iterate through all values, checking them against my unwanted values, and changing them on the fly (and if so, what would this look like. I can't find much about how to handle the ruby 'event' field)? Is there another plugin that easily handles this?

I can do the same things with sub, but you have to list out every field that would need to be changed. If I have 25 different feeds, then that becomes a cumbersome list to maintain.

2

Unless you want to hardcode the names of the fields to check and you don't use a template-based configuration management tool to generate your configuration you need to use a ruby filter.

3

Thanks for the help. Can you clue me in on how this would work?

I saw another answer that looked like what I have below. Would this do the trick?

ruby {
        code => '
            event.to_hash.each { |k, v|
                event[k] = "null" if v == "-" || v == "(empty)"
            }
        '
    }
4

Close. You'll want to assign nil instead of "null".