# Headers in CSV File

**URL:** <https://discuss.elastic.co/t/headers-in-csv-file/161420>\
**Category:** Logstash\
**Created:** [December 18, 2018, 9:48pm UTC](https://discuss.elastic.co/t/headers-in-csv-file/161420 "2018-12-18T21:48:06Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![kb2295](https://avatars.discourse-cdn.com/v4/letter/k/4491bb/32.png) [@kb2295](https://discuss.elastic.co/u/kb2295)\
**Post date:** [December 18, 2018, 9:48pm UTC](https://discuss.elastic.co/t/headers-in-csv-file/161420/1 "2018-12-18T21:48:06Z")

</div>

Hello.

How can I get ELK to ignore the first line in my CSV file (the headers). I keep getting the error:

> [2018-12-18T16:43:18,640][WARN][logstash.outputs.elasticsearch] Failed action. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"delete5", :\_type=\>"logs", :\_routing=\>nil}, 2018-12-18T21:43:18.298Z [lswpffusapn1u.nam.nsroot.net](http://lswpffusapn1u.nam.nsroot.net) Date,Year,Month,Client,Execution Account,Source,SharedBook,I/E/H,FirmId,TraderLogin,Trader,Region,City,Exchange,ProductName,AssetClass,CTM,P&L,CFOXType,TradeType,HandleInst,Manual Fill,Strategy,HFT,Gratis,Sales Region,Legal Entity Region,Lots,OrderCount,FillCount,PerOfLots,PerOfOrders,PerOfFills,exchTraderId,execBrokerCode,tradingPlatform,responsiblePerson], :response=\>{"index"=\>{"\_index"=\>"delete5", "\_type"=\>"logs", "\_id"=\>"AWfDResxZze2tBVL2TrF", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse [Date]", "caused\_by"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Invalid format: "Date""}}}}}

This is what my CSV file looks like:

> Date,Year,Month,Client,Execution Account,Source,SharedBook,I/E/H,FirmId,TraderLogin,Trader,Region,City,Exchange,ProductName,AssetClass,CTM,P&L,CFOXType,TradeType,HandleInst,Manual Fill,Strategy,HFT,Gratis,Sales Region,Legal Entity Region,Lots,OrderCount,FillCount,PerOfLots,PerOfOrders,PerOfFills,exchTraderId,execBrokerCode,tradingPlatform,responsiblePerson  
> "2017-09-20 00:00:00.0","2017","9","XXXX","TTTTT","ABC","FGH","E","XYZ","XYZ","XYZ","XYZ","XYZ","XYZ","XYZ-","ABs","GHI-00000","GHI","H","GHI","GHI","false","GHI","false","false","GHI","GHI","10","1","1","0","0","0","GH5555I","SSB","QQQ","PO0090"

I'm trying to use a .config file to upload the data:

> input {  
> file {  
> path =\> "/tmp/path/fakeMIScsv.csv"  
> start\_position =\> "beginning"  
> sincedb\_path =\>"dev/null"  
> }
> 
> }  
> filter {  
> csv {  
> separator =\> ","  
> columns =\> ["Date","Year","Month","Client","Execution Account","Source","SharedBook","I\_E\_H","FirmId","TraderLogin","Trader","Region","City","Exchange","ProductName","AssetClass","CTM","P\_L","CFOXType","TradeType","HandleInst","Manual Fill","Strategy","HFT","Gratis","Sales Region","Legal Entity Region","Lots","OrderCount","FillCount","PerOfLots","PerOfOrders","PerOfFills","exchTraderId","execBrokerCode","tradingPlatform","responsiblePerson"]  
> }  
> mutate{  
> convert =\> {  
> "Lots" =\> "integer"  
> "OrderCount" =\> "integer"  
> "FillCount" =\> "integer"  
> "PerOfLots" =\> "integer"  
> "PerOfOrders" =\> "integer"  
> "PerOfFills" =\> "integer"  
> }  
> }  
> date {  
> target =\> "Date"  
> match =\> ["Date", "yyyy-MM-dd HH:mm:ss"]  
> }  
> }
> 
> output {  
> elasticsearch {  
> hosts=\> ["abc","xyz","abcd"]  
> index =\> "delete4"  
> user =\> \*\*\*\*\*  
> password =\> \*\*\*\*\*  
> ssl =\> true  
> ssl\_certificate\_verification =\> false  
> cacert =\> '/path/to/cert/ca.crt'  
> }  
> stdout {}  
> }

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 19, 2018, 4:01am UTC](https://discuss.elastic.co/t/headers-in-csv-file/161420/2 "2018-12-19T04:01:47Z")

</div>

Have a look at [https://www.elastic.co/guide/en/logstash/current/plugins-filters-csv.html#plugins-filters-csv-skip\_header](https://www.elastic.co/guide/en/logstash/current/plugins-filters-csv.html#plugins-filters-csv-skip_header)

Otherwise you need a drop with a conditional.

---

<div class="post-metadata">

**Author:** ![StephenGoodall](https://avatars.discourse-cdn.com/v4/letter/s/bb73d2/32.png) [@StephenGoodall](https://discuss.elastic.co/u/StephenGoodall)\
**Post date:** [December 19, 2018, 5:02am UTC](https://discuss.elastic.co/t/headers-in-csv-file/161420/3 "2018-12-19T05:02:34Z")

</div>

Would removing the columns option and adding the "autodetect\_column\_names" option work aswell? Or is that not recommended?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 19, 2018, 5:44am UTC](https://discuss.elastic.co/t/headers-in-csv-file/161420/4 "2018-12-19T05:44:20Z")

</div>

It works fine 🙂

---

<div class="post-metadata">

**Author:** ![kb2295](https://avatars.discourse-cdn.com/v4/letter/k/4491bb/32.png) [@kb2295](https://discuss.elastic.co/u/kb2295)\
**Post date:** [December 19, 2018, 3:51pm UTC](https://discuss.elastic.co/t/headers-in-csv-file/161420/5 "2018-12-19T15:51:07Z")

</div>

Hello,  
Thank you for your response. I get the following error when I try to use that plugin:

> [2018-12-19T10:43:53,193][ERROR][logstash.filters.csv] Unknown setting 'skip\_header' for csv  
> [2018-12-19T10:43:53,203][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Something is wr ong with your configuration."}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 19, 2018, 6:28pm UTC](https://discuss.elastic.co/t/headers-in-csv-file/161420/6 "2018-12-19T18:28:40Z")

</div>

What version are you on?

---

<div class="post-metadata">

**Author:** ![kb2295](https://avatars.discourse-cdn.com/v4/letter/k/4491bb/32.png) [@kb2295](https://discuss.elastic.co/u/kb2295)\
**Post date:** [December 19, 2018, 7:04pm UTC](https://discuss.elastic.co/t/headers-in-csv-file/161420/7 "2018-12-19T19:04:26Z")

</div>

I'm on Version: 5.6.12

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 16, 2019, 7:08pm UTC](https://discuss.elastic.co/t/headers-in-csv-file/161420/8 "2019-01-16T19:08:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
