Removing whitespace from column in csv import

[jay224 (j s)]

I am trying to ingest csv file into ES. The csv file contains undetermined column names in header. I would to eliminate white space from column names. Since I don't know the column names I can't manually supply column names.
e.g. "column name with space" should be renamed to "column_name_with_space". Can I achieve this in logstash?

Thanks

[warkolm (Mark Walkom)]

You could do it with gsub - https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-gsub

[jay224 (j s)]

@warkolm I don't I can use that solution as I will not know the column name in advance. The data in csv files can have different columns.

[paz]

You might need some custom Ruby code to manipulate ad-hoc field names after splitting the original message with CSV.

E.g.

## CSV filter + other stuff
filter {
    ruby  {
        code => "
                hash = event.to_hash
                hash.each{|k,v| event.set(k.gsub(/ /,'_'),v) && event.remove(k) if k.include?(' ')  }
            "
    }
}

It's kinda hacky but gets the job done.