# Autodetect\_column\_names is not working as expected in csv filter pluing

**URL:** <https://discuss.elastic.co/t/autodetect-column-names-is-not-working-as-expected-in-csv-filter-pluing/177161>\
**Category:** Logstash\
**Created:** [April 16, 2019, 10:25pm UTC](https://discuss.elastic.co/t/autodetect-column-names-is-not-working-as-expected-in-csv-filter-pluing/177161 "2019-04-16T22:25:46Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![RohanKumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohankumbhar/32/23649_2.png) [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\
**Post date:** [April 16, 2019, 10:25pm UTC](https://discuss.elastic.co/t/autodetect-column-names-is-not-working-as-expected-in-csv-filter-pluing/177161/1 "2019-04-16T22:25:47Z")

</div>

HI Team,

I'm trying to parse csv files with few rows and trying to auto detect the column names but its not working as expected  
**for example input temp.csv**

> ```
> name,compan,emp,abc,address
> ro han,,,be,myadd
> a b c,234,3 city
> ,ABC CO. LTD.,mycomp,myemp,myabcc, city WEST
> 
> ```

**test.conf**

> input {  
> file {  
> path =\> "/tmp/temp.csv"  
> start\_position =\> "beginning"  
> sincedb\_path =\> "/dev/null"  
> }  
> }  
> filter {
> 
> csv {  
> skip\_header =\> true  
> autodetect\_column\_names =\> true  
> autogenerate\_column\_names =\> true  
> }  
> }  
> output {  
> stdout {codec=\>rubydebug}  
> }

**output**

> {  
> "message" =\> "name,compan,emp,abc,address",  
> "path" =\> "/tmp/temp.csv",  
> "column2" =\> "compan",  
> "myadd" =\> "address",  
> "column3" =\> "emp",  
> "ro han" =\> "name",  
> "@timestamp" =\> 2019-04-16T17:10:14.888Z,  
> "be" =\> "abc",  
> "@version" =\> "1"  
> }  
> {  
> "ro han" =\> "a b c",  
> "@timestamp" =\> 2019-04-16T17:10:14.925Z,  
> "message" =\> "a b c,234,3 city",  
> "@version" =\> "1",  
> "path" =\> "/tmp/temp.csv",  
> "column2" =\> "234",  
> "column3" =\> "3 city"  
> }  
> {  
> "message" =\> ",ABC CO. LTD.,mycomp,myemp,myabcc, city WEST",  
> "path" =\> "/tmp/temp.csv",  
> "column2" =\> "ABC CO. LTD.",  
> "myadd" =\> "myabcc",  
> "column3" =\> "mycomp",  
> "column6" =\> " city WEST",  
> "ro han" =\> nil,  
> "@timestamp" =\> 2019-04-16T17:10:14.926Z,  
> "be" =\> "myemp",  
> "@version" =\> "1"  
> }

Kindly help here to auto detect column names

Thanks,  
Rohan

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 16, 2019, 10:57pm UTC](https://discuss.elastic.co/t/autodetect-column-names-is-not-working-as-expected-in-csv-filter-pluing/177161/2 "2019-04-16T22:57:37Z")

</div>

Have you set "--pipeline.workers 1"? You cannot use multiple worker threads with autodetect\_column\_names because it creates race conditions. Specifically, a second worker thread could parse the second row and use it to set the column names before the first worker thread does so, which appears to be exactly what happened here.

Issues 65 and [72](https://github.com/logstash-plugins/logstash-filter-csv/issues/72) on github are related.

---

<div class="post-metadata">

**Author:** ![RohanKumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohankumbhar/32/23649_2.png) [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\
**Post date:** [April 17, 2019, 10:44am UTC](https://discuss.elastic.co/t/autodetect-column-names-is-not-working-as-expected-in-csv-filter-pluing/177161/3 "2019-04-17T10:44:14Z")

</div>

Thanks @Badger !

Nope i haven't set worker to 1 it was set to default

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 15, 2019, 10:44am UTC](https://discuss.elastic.co/t/autodetect-column-names-is-not-working-as-expected-in-csv-filter-pluing/177161/4 "2019-05-15T10:44:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.