# Multiple workers fail to process CSV files when auto detect column names is set

**URL:** <https://discuss.elastic.co/t/multiple-workers-fail-to-process-csv-files-when-auto-detect-column-names-is-set/283121>\
**Category:** Logstash\
**Created:** [September 2, 2021, 3:55am UTC](https://discuss.elastic.co/t/multiple-workers-fail-to-process-csv-files-when-auto-detect-column-names-is-set/283121 "2021-09-02T03:55:22Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ylmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylmp/32/94063_2.png) [@ylmp](https://discuss.elastic.co/u/ylmp)\
**Post date:** [September 2, 2021, 3:55am UTC](https://discuss.elastic.co/t/multiple-workers-fail-to-process-csv-files-when-auto-detect-column-names-is-set/283121/1 "2021-09-02T03:55:22Z")

</div>

The issue described here is related to CSV filter plugin with autodetect\_column\_names set.  
I experience inconsistent behavior when processing CSV files with multiple workers.  
Seems that the first line in the file is recognized as column names by the CSV filter by one of the workers and any following CSV lines which goes to other workers fail.  
This makes sense to me as each worker has it's own instance of the filter and no global state is shared.

Please confirm this issue/limitation. Logstash has by default multi worker setup and there are no warnings in the CSV filter documentation of issues which would occur as a result.

```auto
input {
  file {
    path => "/input/**/*.csv"    
  }
}

filter {
  
  csv {
      autodetect_column_names => true
      skip_header => true
  } 
...  

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 2, 2021, 1:01pm UTC](https://discuss.elastic.co/t/multiple-workers-fail-to-process-csv-files-when-auto-detect-column-names-is-set/283121/2 "2021-09-02T13:01:26Z")

</div>

> [@ylmp](#):
>
> there are no warnings in the CSV filter documentation

Not so. The [documentation](https://www.elastic.co/guide/en/logstash/current/plugins-filters-csv.html#plugins-filters-csv-autodetect_column_names) for that option says " Logstash pipeline workers must be set to `1` for this option to work".

---

<div class="post-metadata">

**Author:** ![ylmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylmp/32/94063_2.png) [@ylmp](https://discuss.elastic.co/u/ylmp)\
**Post date:** [September 2, 2021, 1:37pm UTC](https://discuss.elastic.co/t/multiple-workers-fail-to-process-csv-files-when-auto-detect-column-names-is-set/283121/3 "2021-09-02T13:37:10Z")

</div>

Missed that,  
Thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 30, 2021, 1:37pm UTC](https://discuss.elastic.co/t/multiple-workers-fail-to-process-csv-files-when-auto-detect-column-names-is-set/283121/4 "2021-09-30T13:37:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.