Removing whitespace from column in csv import

I am trying to ingest csv file into ES. The csv file contains undetermined column names in header. I would to eliminate white space from column names. Since I don't know the column names I can't manually supply column names.
e.g. "column name with space" should be renamed to "column_name_with_space". Can I achieve this in logstash?


You could do it with gsub -

@warkolm I don't I can use that solution as I will not know the column name in advance. The data in csv files can have different columns.

You might need some custom Ruby code to manipulate ad-hoc field names after splitting the original message with CSV.


## CSV filter + other stuff
filter {
    ruby  {
        code => "
                hash = event.to_hash
                hash.each{|k,v| event.set(k.gsub(/ /,'_'),v) && event.remove(k) if k.include?(' ')  }

It's kinda hacky but gets the job done.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.