Reading the heading (1st line) of CSV file through logstash

Elastic Stack Logstash
6

As @BinaryMonkey has suggested you can try the setting the first or both of:

autodetect_column_names => true
autogenerate_column_names => true

See https://github.com/logstash-plugins/logstash-filter-csv/blob/master/lib/logstash/filters/csv.rb#L126
The line of code above will capture the first event seen by the plugin (on all LS restarts) as the column names.

The real pitfall with this is:

  1. Only the very first line of the very first file will be the columns for all other files - because the CSV filter does not keep a map of file_path -> columns internally.
  2. If you have to restart Logstash while it is half way though a file then the first line of that file will not be re-read and the columns will become some arbitrary set of values.
1 Like