Assign a value if field has no value present

Elastic Stack Logstash
1

I am gone a read a csv formated file which has more then 50 field,
by default I want to assign a value "UNKNOWN" to any blank field. how do I easily do it ?

on smaller files I have done
if ![var1] { mutate {update......

but for this long if there is a way to do it for all.

2

You could try

    ruby {
        init => '
            Columns = [ "C1", "C2" ]
        '
        code => '
            Columns.each { |k|
                unless event.get(k)
                    event.set(k, "UNKNOWN")
                end
            }
        '
    }
2 Likes
3

Badger,
can you explain what is columns = ["c1" , "c2"] is doing?

4

Those would be the names of the columns in the CSV file, that is, the set of fields that you want to set to unknown if they are not present in the CSV. I should have mentioned that you will want to set skip_empty_columns on the csv filter so that the fields do not get created if they are empty.

5

perefect I will give this a try next week.

6

Worked. just had to put longgg columns list, alll empty field has value now.

7

Hi quick question this thread again.
Can I do this without explicitly to all the columns. i.e columns = ['*]

I try Star with quote,doublequote, singlequote etc.. none of them worked.

ruby {
    init => '
        Columns = [ "C1", "C2" ]
    '
    code => '
        Columns.each { |k|
            unless event.get(k)
                event.set(k, "UNKNOWN")
            end
        }
    '
}
1 Like
8

The following might work

    ruby {
        code => '
            event.to_hash.each { |k, v|
                if v == nil
                    event.set(k, "UNKNOWN")
                end
            }
        '
    }

I cannot remember why I did not suggest this earlier, but there may have been a reason.

1 Like
9

@Badger

this one did work Thank you

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.