# CSV File re-reading/re-parsing issue 7.10

**URL:** <https://discuss.elastic.co/t/csv-file-re-reading-re-parsing-issue-7-10/276940>\
**Category:** Logstash\
**Created:** [June 24, 2021, 2:09pm UTC](https://discuss.elastic.co/t/csv-file-re-reading-re-parsing-issue-7-10/276940 "2021-06-24T14:09:05Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rajesh\_S](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajesh_s/32/90783_2.png) [@Rajesh\_S](https://discuss.elastic.co/u/Rajesh_S)\
**Post date:** [June 24, 2021, 2:09pm UTC](https://discuss.elastic.co/t/csv-file-re-reading-re-parsing-issue-7-10/276940/1 "2021-06-24T14:09:05Z")

</div>

I am trying to re-read a csv file from the beginning using sincedb set to '/dev/null'. logstash is started using --config.reload.automatic. All entries are blank when the file is re-read. This is not consistent, some times data is read, sometimes not. to trigger re-parsing, i just open csvread.config and add/remove spaces at the end.

command line : bin/logstash -f csvread.config --config.reload.automatic

Configuration file :

```auto
input {
  file {
    path => "/home/whoami/result6.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  } 
}

filter {
    csv {
      separator => ","
	autodetect_column_names => true
	autogenerate_column_names => true	  
    }
    mutate {
		add_field => { "out_timestamp" => "%{@timestamp}"} 
	}
    mutate {
		rename => { 
          "active" => "ACTIVE"
          "state_name" => "STATE" } 
	}
    mutate {
		update => { } 
	}
  ruby {
        code => 
          'event.set("OTH_MAPPING",[])'
	}
    prune {
        whitelist_names => ["out_timestamp", "^ACTIVE$", "^STATE$", "^OTH_MAPPING$"]
    }	
}
output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "csv_read"
  }
 }

```

result6.csv :  
sno,col1,col2,col3,col4,col5,col6,col7,col8,col9,col10  
2,column info,75,5175,5038,62,88,5190,5040,62,35  
1,column info 1,18666,921906,895949,7291,20954,925401,897147,7300,28

sometimes csv is read as below which is an issue :

```auto
{
      "OTH_MAPPING" => [],
    "out_timestamp" => "2021-06-24T12:22:36.555Z"
}
{
      "OTH_MAPPING" => [],
    "out_timestamp" => "2010-06-24T12:22:36.555Z"
}
{
      "OTH_MAPPING" => [],
    "out_timestamp" => "2010-06-24T12:22:36.556Z"
}

```

ideally it should have read as :

```auto
{
        "ACTIVE" => "75",
      "OTH_MAPPING" => [],
    "out_timestamp" => "2010-06-24T12:22:18.434Z",
         "STATE" => "column0"
}
{
        "ACTIVE" => "32",
      "OTH_MAPPING" => [],
    "out_timestamp" => "2010-06-24T12:22:18.435Z",
         "STATE" => "Column1"
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 24, 2021, 3:53pm UTC](https://discuss.elastic.co/t/csv-file-re-reading-re-parsing-issue-7-10/276940/2 "2021-06-24T15:53:09Z")

</div>

What are your settings for pipeline.workers and pipeline.ordered?

---

<div class="post-metadata">

**Author:** ![Rajesh\_S](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajesh_s/32/90783_2.png) [@Rajesh\_S](https://discuss.elastic.co/u/Rajesh_S)\
**Post date:** [June 25, 2021, 5:29am UTC](https://discuss.elastic.co/t/csv-file-re-reading-re-parsing-issue-7-10/276940/3 "2021-06-25T05:29:50Z")

</div>

> [@Badger](#):
>
> pipeline.workers

pipeline.workers is not set.  
pipeline.ordered is set to auto in once case and is not set in other scenario, its not working in both the cases.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 25, 2021, 3:36pm UTC](https://discuss.elastic.co/t/csv-file-re-reading-re-parsing-issue-7-10/276940/4 "2021-06-25T15:36:07Z")

</div>

The [documentation](https://www.elastic.co/guide/en/logstash/current/plugins-filters-csv.html#plugins-filters-csv-autodetect_column_names) says that pipeline.workers must be set to 1 for autodetect\_column\_names to work.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 23, 2021, 3:37pm UTC](https://discuss.elastic.co/t/csv-file-re-reading-re-parsing-issue-7-10/276940/5 "2021-07-23T15:37:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.