# Write csv to elastic search by using logstash

**URL:** <https://discuss.elastic.co/t/write-csv-to-elastic-search-by-using-logstash/76338>\
**Category:** Logstash\
**Created:** [February 24, 2017, 2:45am UTC](https://discuss.elastic.co/t/write-csv-to-elastic-search-by-using-logstash/76338 "2017-02-24T02:45:34Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ximeng\_Zhao](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ximeng_zhao/32/15775_2.png) [@Ximeng\_Zhao](https://discuss.elastic.co/u/Ximeng_Zhao)\
**Post date:** [February 24, 2017, 2:45am UTC](https://discuss.elastic.co/t/write-csv-to-elastic-search-by-using-logstash/76338/1 "2017-02-24T02:45:34Z")

</div>

Hello All,

I want to write a csv file to the elasticsearch, but after running ./logstash -f logstash.conf, es and kibana, you can check the indices is created properly but only with ten records. It doesn't import all data to the ES. Can anyone tell me where's the problem and how to fix it? Thanks so much.

Here's my logstach.conf:

input {  
file {  
path =\> "/Users/simon/Desktop/simon.csv"  
start\_position =\> "beginning"  
sincedb\_path =\> "/dev/null"  
}  
}  
filter {  
csv {  
separator =\> ","  
#Date,Open,High,Low,Close,Volume (BTC),Volume (Currency),Weighted Price  
columns =\> ["Date","Open","High","Low","Close","Volume (BTC)", "Volume (Currency)" ,"Weighted Price"]  
}  
}  
output {  
elasticsearch {  
hosts =\> "[http://localhost:9200](http://localhost:9200)"  
index =\> "bitcoin-prices"  
}  
stdout {}  
}

Here's one record looks like on kibana when u check the indice:  
{  
"\_index": "bitcoin-prices",  
"\_type": "logs",  
"\_id": "AVpt-S3DeuqB-XFNJ-4m",  
"\_score": 1,  
"\_source": {  
"High": "139.0",  
"Volume (BTC)": "6405.77673665",  
"Volume (Currency)": "878670.987072",  
"Weighted Price": "137.168531342",  
"message": "2013-10-07,137.01002,139.0,135.12,135.80001,6405.77673665,878670.987072,137.168531342",  
"Date": "2013-10-07",  
"tags": [],  
"Open": "137.01002",  
"path": "/Users/simon/Desktop/simon.csv",  
"@timestamp": "2017-02-24T02:35:30.624Z",  
"Low": "135.12",  
"@version": "1",  
"host": "simons-MacBook-Air.local",  
"Close": "135.80001"  
}  
},

Here's the debug information from ES:  
[DEBUG][o.e.a.b.TransportShardBulkAction] [ZUWG2gq] [bitcoin-prices][2] failed to execute bulk item (index) index {[bitcoin-prices][logs][AVpt-S3GeuqB-XFNJ-6L], source[{"High":"High","Volume (BTC)":"Volume (BTC)","Volume (Currency)":"Volume (Currency)","Weighted Price":"Weighted Price","message":"Date,Open,High,Low,Close,Volume (BTC),Volume (Currency),Weighted Price","Date":"Date","tags":[],"Open":"Open","path":"/Users/simon/Desktop/simon.csv","@timestamp":"2017-02-24T02:35:29.743Z","Low":"Low","@version":"1","host":"simons-MacBook-Air.local","Close":"Close"}]}  
org.elasticsearch.index.mapper.MapperParsingException: failed to parse [Date]

Thanks again

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 24, 2017, 2:47am UTC](https://discuss.elastic.co/t/write-csv-to-elastic-search-by-using-logstash/76338/2 "2017-02-24T02:47:41Z")

</div>

That error is likely because it's trying to treat the header line as a record.

How many document short are you?

---

<div class="post-metadata">

**Author:** ![Ximeng\_Zhao](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ximeng_zhao/32/15775_2.png) [@Ximeng\_Zhao](https://discuss.elastic.co/u/Ximeng_Zhao)\
**Post date:** [February 24, 2017, 2:52am UTC](https://discuss.elastic.co/t/write-csv-to-elastic-search-by-using-logstash/76338/3 "2017-02-24T02:52:59Z")

</div>

what do you mean "document short"? Thx

---

<div class="post-metadata">

**Author:** ![Ximeng\_Zhao](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ximeng_zhao/32/15775_2.png) [@Ximeng\_Zhao](https://discuss.elastic.co/u/Ximeng_Zhao)\
**Post date:** [February 24, 2017, 3:01am UTC](https://discuss.elastic.co/t/write-csv-to-elastic-search-by-using-logstash/76338/4 "2017-02-24T03:01:54Z")

</div>

I got this error 🙂

Caused by: org.elasticsearch.ElasticsearchParseException: failed to parse date field [2015-05-01] with format [date\_time]

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 24, 2017, 4:03am UTC](https://discuss.elastic.co/t/write-csv-to-elastic-search-by-using-logstash/76338/5 "2017-02-24T04:03:32Z")

</div>

You mentioned not all the data is in ES, so how much is missing?

---

<div class="post-metadata">

**Author:** ![Ximeng\_Zhao](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ximeng_zhao/32/15775_2.png) [@Ximeng\_Zhao](https://discuss.elastic.co/u/Ximeng_Zhao)\
**Post date:** [February 24, 2017, 4:10am UTC](https://discuss.elastic.co/t/write-csv-to-elastic-search-by-using-logstash/76338/6 "2017-02-24T04:10:24Z")

</div>

1320 rows with 1310 missing, thanks,

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 24, 2017, 4:10am UTC](https://discuss.elastic.co/t/write-csv-to-elastic-search-by-using-logstash/76338/7 "2017-03-24T04:10:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.