Logstash-Filter-CSV plugin clarification (skip header and define column )

[Ajay1 (sridhar)]

Hi All,

I recently upgraded Logstash from 5.0 to 5.3 and using Filter-csv plugin for some of the reports .

My configuration goes fine without any changes but in Kibana i could not get the data in column specific (expanded data set) even though i have defined it in logstash filter csv configuration.

All my columns are listed in single line in the field called "message" .
Strangely same configuration is used for different report is working fine with expanded data set and the only difference with that separator is semicolon.

Below config takes all the column data in a single line in "message" which is unexpected or not the intended one i want to have .

 filter {
      if "report_rejects" in [app] {
     csv {
     columns => ['ID','Rejection reason','A','B','C']
     separator => ","
     skip_empty_columns => "true"
     remove_field => ["message"]
     }
    }
 }

also in Index template i have defined all the columns with type and their fields properties respectively.
Is it something with the version of plugin or index template .

2. Second query , how can i skip this header of column itself , because everytime one entry is allocated for header and it is confusing with dashboard when i want to make a count the number of records .