# Csv logststash

**URL:** <https://discuss.elastic.co/t/csv-logststash/120393>\
**Category:** Logstash\
**Created:** [February 19, 2018, 4:39am UTC](https://discuss.elastic.co/t/csv-logststash/120393 "2018-02-19T04:39:32Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pororo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pororo/32/27711_2.png) [@Pororo](https://discuss.elastic.co/u/Pororo)\
**Post date:** [February 19, 2018, 4:39am UTC](https://discuss.elastic.co/t/csv-logststash/120393/1 "2018-02-19T04:39:32Z")

</div>

I was trying to import csv file to ES through logstash. But when I looked through the data in kibana, I just found that the first line of csv file(attributes of data) is accidentally recognized as data instance instead of columns.

This is the screenshot of kibana.

 ![03](https://us1.discourse-cdn.com/elastic/original/3X/6/3/63d4b0cf2ab33f91a9f3ea4c5799e4ba0c5ebf4c.png)

These are my csv file and config file.

```
ID,TweetDate,Tweet Time,Brand Name,Original Tweet URL,Tweet Text only,Tweet Image,Tweet URL,Tweet Image + URL,Tweet Video,Tweet Video + URL,1.Brand Awareness,1.Type,2.CSR,2.Type,3.Customer Service,3.Type,4.Engagement,4.Type,5.Product Awareness,5.Type,6.Promotional,6.Type,7.Seasonal,7.Type,Appeal-Transformational,Appeal-Informational,# Likes,# Mentions,# Retweets,TweetDateTime
1,2018/2/15,6:49am,WebMD,https://twitter.com/WebMD/status/964152218059464705,0,0,0,1,0,0,0,0,1,1,1,4,1,4,0,0,0,0,0,0,0,1,17,0,19,2018-2-15 6:49
2,2018/2/15,2:00am,WebMD,https://twitter.com/WebMD/status/964076827852582912,0,0,0,1,0,0,0,0,1,1,1,4,1,1,0,0,0,0,1,3,0,1,77,3,48,2018-2-15 2:00
3,2018/2/14,9:00am,WebMD,https://twitter.com/WebMD/status/963820299010629635,0,0,0,0,1,0,0,0,0,0,0,0,1,1,0,0,0,0,1,2,1,0,43,0,16,2018-2-14 9:00
4,2018/2/15,1:29pm,WebMD,https://twitter.com/WebMD/status/964250343294095360,0,0,0,1,0,0,0,0,1,1,1,4,0,0,0,0,0,0,1,3,0,1,4,4,25,2018-2-15 13:29

input {
	file {
		path => "/Users/apple/Desktop/ITM444/logstash_TW/webmd_twitter.csv"
		start_position => "beginning"
		sincedb_path => "/dev/null"
	}
}
filter {
	csv {
		separator => ","
		columns => ["ID", "TweetDate", "Tweet Time", "Brand Name", "Original Tweet URL", "Tweet Text only", "Tweet Image", "Tweet URL", "Tweet Image + URL", "Tweet Video", "Tweet Video + URL", "1.Brand Awareness", "1.Type", "2.CSR", "2.Type", "3.Customer Service", "3.Type", "4.Engagement", "4.Type", "5.Product Awareness", "5.Type", "6.Promotional", "6.Type", "7.Seasonal", "7.Type", "Appeal-Transformational", "Appeal-Informational", "# Likes", "# Mentions", "# Retweets", "TweetDateTime"]
	}
	mutate {convert => ["# Likes", "integer"]}
	mutate {convert => ["# Mentions", "integer"]}
	mutate {convert => ["# Retweets", "integer"]}
	
	date{
		match => ["TweetDateTime", "yyyy-MM-dd HH:mm"]
	}

}
output {
	elasticsearch {
		hosts => "localhost:9200"
		index => "webmd"
		document_type => "twitter"
	}
	stdout { codec => rubydebug }
}
```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 19, 2018, 4:48am UTC](https://discuss.elastic.co/t/csv-logststash/120393/2 "2018-02-19T04:48:00Z")

</div>

I think you are missing the question here, what would you like assistance with 🙂

---

<div class="post-metadata">

**Author:** ![Pororo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pororo/32/27711_2.png) [@Pororo](https://discuss.elastic.co/u/Pororo)\
**Post date:** [February 19, 2018, 4:50am UTC](https://discuss.elastic.co/t/csv-logststash/120393/3 "2018-02-19T04:50:24Z")

</div>

As you can see, my csv file actually has 4 data items and one line for the column.

But when I import the file, it is recognized as 5 data items (including the header line).

Is there any way to avoid this?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 19, 2018, 5:02am UTC](https://discuss.elastic.co/t/csv-logststash/120393/4 "2018-02-19T05:02:50Z")

</div>

Ok, then [https://www.elastic.co/guide/en/logstash/current/plugins-filters-csv.html#plugins-filters-csv-autodetect\_column\_names](https://www.elastic.co/guide/en/logstash/current/plugins-filters-csv.html#plugins-filters-csv-autodetect_column_names) should do what you want.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 19, 2018, 5:02am UTC](https://discuss.elastic.co/t/csv-logststash/120393/5 "2018-03-19T05:02:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.