# Handle Blank data in CSV - Ingest By Using Logstash

**URL:** <https://discuss.elastic.co/t/handle-blank-data-in-csv-ingest-by-using-logstash/131867>\
**Category:** Logstash\
**Created:** [May 15, 2018, 7:30am UTC](https://discuss.elastic.co/t/handle-blank-data-in-csv-ingest-by-using-logstash/131867 "2018-05-15T07:30:26Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![nandan\_bigdata](https://avatars.discourse-cdn.com/v4/letter/n/f07891/32.png) [@nandan\_bigdata](https://discuss.elastic.co/u/nandan_bigdata)\
**Post date:** [May 15, 2018, 7:30am UTC](https://discuss.elastic.co/t/handle-blank-data-in-csv-ingest-by-using-logstash/131867/1 "2018-05-15T07:30:27Z")

</div>

Hi,  
I have CSV file as below

> col1,col2,col3,col4  
> A,B,C,D  
> ,,E,F  
> ,G,H  
> ,,,I  
> J,K,,,

More then 10 milion records,  
I am using below config code as :-

> input {  
> file {  
> path =\> "/home/nandan/data/data.csv"  
> start\_position =\> "beginning"  
> sincedb\_path =\> "/dev/null"  
> }  
> }  
> filter {  
> csv {  
> separator =\> ","  
> columns =\> ["col1","col2","col3","col4"]  
> }  
> mutate {convert =\> ["col1","integer"] }  
> mutate {convert =\> ["col3","integer"] }  
> }  
> output {  
> elasticsearch {  
> hosts =\> "localhost"  
> index =\> "hotel"  
> document\_type =\> "rooms"  
> }  
> stdout{}  
> }

but only half data are able to index into elasticsearch,  
Error is :-

> [ERROR] 2018-05-15 14:43:56.918 [LogStash::Runner] Logstash - org.jruby.exceptions.ThreadKill  
> [WARN] 2018-05-15 14:43:56.939 [Ruby-0-Thread-9@[main]\>worker0: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:385] csv - Error parsing csv {:field=\>"message", :source=\>"",34103,,UNITED STATES - USA,26.1858,-81.799\r", :exception=\>#\<CSV::MalformedCSVError: Unclosed quoted field on line 1.\>}  
> [WARN] 2018-05-15 14:43:56.948 [Ruby-0-Thread-9@[main]\>worker0: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:385] csv - Error parsing csv {:field=\>"message", :source=\>"81512,supplier2,Chinatown Hotel,YAOWARAJ SUMPUNTAWONG 526,"Bangkok", :exception=\>#\<CSV::MalformedCSVError: Unclosed quoted field on line 1.\>}

and only half data indexed.. Please tell me why is it happening.

Thanks

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 15, 2018, 1:50pm UTC](https://discuss.elastic.co/t/handle-blank-data-in-csv-ingest-by-using-logstash/131867/2 "2018-05-15T13:50:07Z")

</div>

> [@nandan\_bigdata](#):
>
> source=\>"",34103,,UNITED STATES - USA,26.1858,-81.799\r"

You have a line that starts with a double quote, and does not have a closing double quote. That is an unclosed quoted field, so the csv filter fails to parse it. Same for the second line, there is no closing quote at the end of Bangkok.

However, that should not prevent the data being indexed, it should just be missing the fields you want.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 12, 2018, 1:50pm UTC](https://discuss.elastic.co/t/handle-blank-data-in-csv-ingest-by-using-logstash/131867/3 "2018-06-12T13:50:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.