# Multiple csv with different columns

**URL:** <https://discuss.elastic.co/t/multiple-csv-with-different-columns/271794>\
**Category:** Logstash\
**Created:** [April 30, 2021, 2:41pm UTC](https://discuss.elastic.co/t/multiple-csv-with-different-columns/271794 "2021-04-30T14:41:50Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![dpb\_2000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dpb_2000/32/88027_2.png) [@dpb\_2000](https://discuss.elastic.co/u/dpb_2000)\
**Post date:** [April 30, 2021, 2:41pm UTC](https://discuss.elastic.co/t/multiple-csv-with-different-columns/271794/1 "2021-04-30T14:41:50Z")

</div>

Hi, I am trying to parse multiple csv with different columns based on **autodetect\_column\_names =\> true** argument. The main problem is that when I upload the data the columns get repeated and only the columns of the first csv are taking into account.  
Does anyone know why this is happening and how to solve it?

```auto
input {
  file {
    path => "/home/user/Documents/data/*.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  } 
}
filter { 
	csv {
		separator => ","
		skip_header => "true"
		autodetect_column_names => true
		autogenerate_column_names => true
	}
}
output {
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "final_index"
  }
stdout {}
}

```

Thanks

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 30, 2021, 3:53pm UTC](https://discuss.elastic.co/t/multiple-csv-with-different-columns/271794/2 "2021-04-30T15:53:45Z")

</div>

When you set autodetect\_column\_names true the very first event that the filter sees is parsed as column headers. Once that is done it is never re-done. If you have multiple files with different columns the only way to use autodetect\_column\_names is to pass them through different csv filters. See [this](https://discuss.elastic.co/t/logstash-cant-recognize-csv-columns/196396/2) thread.

---

<div class="post-metadata">

**Author:** ![dpb\_2000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dpb_2000/32/88027_2.png) [@dpb\_2000](https://discuss.elastic.co/u/dpb_2000)\
**Post date:** [April 30, 2021, 4:39pm UTC](https://discuss.elastic.co/t/multiple-csv-with-different-columns/271794/3 "2021-04-30T16:39:23Z")

</div>

Okey, but what if I don´t know any conditions about them?  
I mean, what I want is to iterate as a loop in those csv, _without using any if condition._  
Do you have any idea?

Thanks

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 30, 2021, 5:04pm UTC](https://discuss.elastic.co/t/multiple-csv-with-different-columns/271794/4 "2021-04-30T17:04:57Z")

</div>

I do not know of any way to treat different events differently without using a conditional.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 28, 2021, 5:05pm UTC](https://discuss.elastic.co/t/multiple-csv-with-different-columns/271794/5 "2021-05-28T17:05:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.