# Logstash 5.1.1 gets hanged while reading csv

**URL:** <https://discuss.elastic.co/t/logstash-5-1-1-gets-hanged-while-reading-csv/69172>\
**Category:** Logstash\
**Created:** [December 15, 2016, 2:02pm UTC](https://discuss.elastic.co/t/logstash-5-1-1-gets-hanged-while-reading-csv/69172 "2016-12-15T14:02:10Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hassan\_Nashrah](https://avatars.discourse-cdn.com/v4/letter/h/51bf81/32.png) [@Hassan\_Nashrah](https://discuss.elastic.co/u/Hassan_Nashrah)\
**Post date:** [December 15, 2016, 2:02pm UTC](https://discuss.elastic.co/t/logstash-5-1-1-gets-hanged-while-reading-csv/69172/1 "2016-12-15T14:02:10Z")

</div>

when i run logstash. it starts read the csv send data to elasticsearch as well. but it hangs in the middle.

logs show no error

```
    input {
     file {
        path => "/opt/logstash-5.1.1/bin/01.12.2016.csv"
        start_position => "beginning"
       sincedb_path => "/dev/null"
      }
    }
    filter {
      csv {
          separator => ","
         columns => ["departureDate","flightNo","passportNo"]
      }

    if [departureDate] == "departureDate" { drop{ } }

    }
    output {
       elasticsearch {
         hosts => "http://192.168.2.26:9200"
         index => "dollar"
      }
    stdout { }
    }

```

Below shows the console output while reading the file

2016-12-15T12:43:10.216Z localhost.localdomain 2016-12-01,UL116,E0100000  
2016-12-15T12:43:10.216Z localhost.localdomain 2016-12-01,UL116,E0000000  
2016-12-15T12:43:10.216Z localhost.localdomain 2016-12-01,UL116,E00000000  
2016-12-15T12:43:10.216Z localhost.localdomain 2016-12-01,UL116,NC000000  
2016-12-15T12:43:10.216Z localhost.localdomain 2016-12-01,UL116,LA0000000  
2016-12-15T12:43:10.217Z localhost.localdomain 2016-12-01,UL116,L00000000  
2016-12-15T12:43:10.217Z localhost.localdomain 2016-12-01,UL116,E00000000

checked cpu , overload its fine,,

i am scratching my head now

---

<div class="post-metadata">

**Author:** ![Hassan\_Nashrah](https://avatars.discourse-cdn.com/v4/letter/h/51bf81/32.png) [@Hassan\_Nashrah](https://discuss.elastic.co/u/Hassan_Nashrah)\
**Post date:** [December 15, 2016, 3:22pm UTC](https://discuss.elastic.co/t/logstash-5-1-1-gets-hanged-while-reading-csv/69172/2 "2016-12-15T15:22:38Z")

</div>

when i run in --verbose --debug

2016-12-15T15:25:54.356Z localhost.localdomain 2010-08-14,0.0655,0.0695,0.0645,0.067,4404.2,295.3084789,0.067051559625  
2016-12-15T15:25:54.356Z localhost.localdomain 2010-08-13,0.0665,0.068,0.0645,0.0645,3591.77,233.8004368,0.065093376469  
2016-12-15T15:25:54.357Z localhost.localdomain 2010-08-12,0.068,0.07,0.06141,0.07,2062.31,134.4872288,0.065211936518  
2016-12-15T15:25:54.357Z localhost.localdomain 2010-08-11,0.06,0.07541,0.06,0.067,14061.18,1015.6416825,0.07223018854  
2016-12-15T15:25:54.357Z localhost.localdomain 2010-08-10,0.068,0.0709,0.06651,0.07,1310.39,88.8736329,0.067822276498  
2016-12-15T15:25:54.370Z localhost.localdomain 2010-08-09,0.06091,0.0735,0.0593,0.071,13631.09,886.9327103,0.065066895626  
2016-12-15T15:25:54.371Z localhost.localdomain 2010-08-08,0.059,0.061,0.059,0.0609,2201.0,132.5954,0.060243253067  
2016-12-15T15:25:54.371Z localhost.localdomain 2010-08-07,0.0608,0.0622,0.059,0.059,2619.0,157.3408,0.060076670485  
2016-12-15T15:25:54.371Z localhost.localdomain 2010-08-06,0.062,0.0624,0.0607,0.0623,1395.0,85.908,0.061582795699  
2016-12-15T15:25:54.372Z localhost.localdomain 2010-08-05,0.0581,0.061,0.058,0.061,5034.07,303.609091,0.06031086  
2016-12-15T15:25:54.372Z localhost.localdomain 2010-08-04,0.06229,0.06231,0.057,0.057,3494.0,210.765111,0.060322012307  
2016-12-15T15:25:54.374Z localhost.localdomain 2010-08-03,0.06,0.065,0.059,0.06,9821.46,606.0456913,0.061706272927  
2016-12-15T15:25:54.374Z localhost.localdomain 2010-08-02,0.0627,0.0633,0.06,0.06,3599.0,221.20058,0.061461678244  
2016-12-15T15:25:54.377Z localhost.localdomain 2010-08-01,0.061,0.065,0.06,0.0611,2601.0,162.6451925,0.06253179258  
2016-12-15T15:25:54.377Z localhost.localdomain 2010-07-31,0.065,0.06889,0.056,0.06785,4022.25,243.898458,0.060637319411  
2016-12-15T15:25:54.378Z localhost.localdomain 2010-07-30,0.0698,0.0698,0.0582,0.0627,3020.85,198.5299149,0.065719885099  
2016-12-15T15:25:54.378Z localhost.localdomain 2010-07-29,0.0597,0.0699,0.0571,0.0699,8058.49,528.3183128,0.065560460185  
2016-12-15T15:25:54.379Z localhost.localdomain 2010-07-28,0.06,0.062,0.054,0.0589,4390.29,255.759194,0.058255649171  
2016-12-15T15:25:54.380Z localhost.localdomain 2010-07-27,0.053,0.0605,0.053,0.06,3373.69,196.91835,0.058368833532  
2016-12-15T15:25:54.380Z localhost.localdomain 2010-07-26,0.05,0.056,0.05,0.056,877.0,46.907,0.053485746864  
2016-12-15T15:25:54.380Z localhost.localdomain 2010-07-25,0.05545,0.05941,0.0505,0.0505,1551.48,85.062975,0.054826987779  
2016-12-15T15:25:54.381Z localhost.localdomain 2010-07-24,0.06161,0.06161,0.05049,0.05454,496.32,26.7277095,0.053851768013  
2016-12-15T15:25:54.381Z localhost.localdomain 2010-07-23,0.0505,0.06767,0.0505,0.06262,2402.5,141.0650511,0.058715942185  
2016-12-15T15:25:54.382Z localhost.localdomain 2010-07-22,0.07921,0.08181,0.0505,0.0505,2160.0,129.78428,0.060085314815  
2016-12-15T15:25:54.382Z localhost.localdomain 2010-07-21,0.07425,0.07921,0.06634,0.07921,575.0,42.2622,0.073499478261  
2016-12-15T15:25:54.383Z localhost.localdomain 2010-07-20,0.08181,0.08181,0.07426,0.07474,262.0,20.58952,0.078585954198  
2016-12-15T15:25:54.383Z localhost.localdomain 2010-07-19,0.0909,0.09307,0.07723,0.0808,574.0,49.6613,0.086517944251  
2016-12-15T15:25:54.383Z localhost.localdomain 2010-07-18,0.05941,0.08585,0.05941,0.08584,75.01,5.0919941,0.06788420344  
2016-12-15T15:25:54.384Z localhost.localdomain 2010-07-17,0.04951,0.04951,0.04951,0.04951,20.0,0.9902,0.04951  
[2016-12-15T20:25:55,364][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created\_at=\>2016-12-15 20:25:55 +0500}  
[2016-12-15T20:25:55,419][DEBUG][logstash.pipeline] Pushing flush onto pipeline  
[2016-12-15T20:25:56,368][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created\_at=\>2016-12-15 20:25:56 +0500}  
[2016-12-15T20:25:57,374][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created\_at=\>2016-12-15 20:25:57 +0500}  
[2016-12-15T20:25:58,377][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created\_at=\>2016-12-15 20:25:58 +0500}  
[2016-12-15T20:25:59,380][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created\_at=\>2016-12-15 20:25:59 +0500}  
[2016-12-15T20:26:00,389][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created\_at=\>2016-12-15 20:26:00 +0500}  
[2016-12-15T20:26:00,423][DEBUG][logstash.pipeline] Pushing flush onto pipeline  
[2016-12-15T20:26:01,395][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created\_at=\>2016-12-15 20:26:01 +0500}  
[2016-12-15T20:26:02,400][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created\_at=\>2016-12-15 20:26:02 +0500}  
[2016-12-15T20:26:03,402][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created\_at=\>2016-12-15 20:26:03 +0500}  
[2016-12-15T20:26:04,410][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created\_at=\>2016-12-15 20:26:04 +0500}  
[2016-12-15T20:26:05,413][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created\_at=\>2016-12-15 20:26:05 +0500}  
[2016-12-15T20:26:05,423][DEBUG][logstash.pipeline] Pushing flush onto pipeline  
[2016-12-15T20:26:06,422][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created\_at=\>2016-12-15 20:26:06 +0500}  
[2016-12-15T20:26:07,426][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created\_at=\>2016-12-15 20:26:07 +0500}  
[2016-12-15T20:26:08,419][DEBUG][logstash.inputs.file] \_globbed\_files: /opt/logstash-5.0.2/bin/1.csv: glob is: ["/opt/logstash-5.0.2/bin/1.csv"]  
[2016-12-15T20:26:08,459][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers

---

<div class="post-metadata">

**Author:** ![Hassan\_Nashrah](https://avatars.discourse-cdn.com/v4/letter/h/51bf81/32.png) [@Hassan\_Nashrah](https://discuss.elastic.co/u/Hassan_Nashrah)\
**Post date:** [December 20, 2016, 9:36am UTC](https://discuss.elastic.co/t/logstash-5-1-1-gets-hanged-while-reading-csv/69172/3 "2016-12-20T09:36:11Z")

</div>

fixed it, there was an empty line in the csv file, by default the file exported from the mysql has an empty line in the bottom, backspace it and it fixes the issue..

took me alot of time to figure it finally it was an empty line 😡

ELK needs to address it as if we automating csv read in a directory removing the empty line should not be a manual process,

also i found that it gives csv parse failure for some records.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 17, 2017, 9:36am UTC](https://discuss.elastic.co/t/logstash-5-1-1-gets-hanged-while-reading-csv/69172/4 "2017-01-17T09:36:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.