# Not able to read large csv file in logstash

**URL:** <https://discuss.elastic.co/t/not-able-to-read-large-csv-file-in-logstash/95987>\
**Category:** Logstash\
**Created:** [August 5, 2017, 5:06pm UTC](https://discuss.elastic.co/t/not-able-to-read-large-csv-file-in-logstash/95987 "2017-08-05T17:06:22Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![developer\_apk](https://avatars.discourse-cdn.com/v4/letter/d/59ef9b/32.png) [@developer\_apk](https://discuss.elastic.co/u/developer_apk)\
**Post date:** [August 5, 2017, 5:06pm UTC](https://discuss.elastic.co/t/not-able-to-read-large-csv-file-in-logstash/95987/1 "2017-08-05T17:06:22Z")

</div>

I am using below config to read large file of 1gb size having around 5m records, it is not giving any output or errors

input {

file {  
path =\> "/filepath/filename.txt"  
}  
}  
filter {  
csv {  
separator =\> "|"  
}  
}  
output {  
stdout {  
}  
}

I tried keeping only 10 records in file, it is working fine  
I tried with --debug option, found below log with 10 records file  
[2017-08-05T11:41:15,011][DEBUG][logstash.inputs.file] \_globbed\_files:  
[2017-08-05T11:41:15,018][DEBUG][logstash.inputs.file] \_discover\_file:  
[2017-08-05T11:41:15,023][DEBUG][logstash.inputs.file] \_open\_file:

with 1gb file only below log was coming (discover\_file and open\_file is not coming up)  
[2017-08-05T11:41:15,011][DEBUG][logstash.inputs.file] \_globbed\_files:

I tried even deleting all sincedb files

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 5, 2017, 10:30pm UTC](https://discuss.elastic.co/t/not-able-to-read-large-csv-file-in-logstash/95987/2 "2017-08-05T22:30:29Z")

</div>

Given it's 1GB it's probably reading the file, how long did you wait?

---

<div class="post-metadata">

**Author:** ![developer\_apk](https://avatars.discourse-cdn.com/v4/letter/d/59ef9b/32.png) [@developer\_apk](https://discuss.elastic.co/u/developer_apk)\
**Post date:** [August 8, 2017, 8:16pm UTC](https://discuss.elastic.co/t/not-able-to-read-large-csv-file-in-logstash/95987/3 "2017-08-08T20:16:06Z")

</div>

1 hour

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 5, 2017, 8:16pm UTC](https://discuss.elastic.co/t/not-able-to-read-large-csv-file-in-logstash/95987/4 "2017-09-05T20:16:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.