Multiple csv with different columns

Elastic Stack Logstash
1

Hi, I am trying to parse multiple csv with different columns based on autodetect_column_names => true argument. The main problem is that when I upload the data the columns get repeated and only the columns of the first csv are taking into account.
Does anyone know why this is happening and how to solve it?

input {
  file {
    path => "/home/user/Documents/data/*.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  } 
}
filter { 
	csv {
		separator => ","
		skip_header => "true"
		autodetect_column_names => true
		autogenerate_column_names => true
	}
}
output {
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "final_index"
  }
stdout {}
}

Thanks

2

When you set autodetect_column_names true the very first event that the filter sees is parsed as column headers. Once that is done it is never re-done. If you have multiple files with different columns the only way to use autodetect_column_names is to pass them through different csv filters. See this thread.

3

Okey, but what if I don´t know any conditions about them?
I mean, what I want is to iterate as a loop in those csv, without using any if condition.
Do you have any idea?

Thanks

4

I do not know of any way to treat different events differently without using a conditional.