# CSV data upload problem

**URL:** <https://discuss.elastic.co/t/csv-data-upload-problem/156956>\
**Category:** Logstash\
**Created:** [November 16, 2018, 12:28am UTC](https://discuss.elastic.co/t/csv-data-upload-problem/156956 "2018-11-16T00:28:22Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![f24b83985b804d90965e](https://avatars.discourse-cdn.com/v4/letter/f/46a35a/32.png) [@f24b83985b804d90965e](https://discuss.elastic.co/u/f24b83985b804d90965e)\
**Post date:** [November 16, 2018, 12:28am UTC](https://discuss.elastic.co/t/csv-data-upload-problem/156956/1 "2018-11-16T00:28:22Z")

</div>

hi there.  
I Have 2 issue about Logstash. but first, I let you know that test base.

- Windows 10
- Search, kibana, Logstash v6.2.2

1. CSV data upload  
it is work in v6.2.2 family. but 6.4 or 6.5.0 doesn't work in same method.  
I found something in the config file.  
**input { file { sincedb\_path =\> "/dev/null"**  
it is working in v6.2.2  
but, other higher version said, 'not found C:\dev\null'  
then I added '#' at that keyword. no error and no upload.  
what can I do?

2. replace @timestamp  
i create log data with .CSV and it have timestamp itself.  
I want replace @timestamp.  
but @timestamp logging what I execute logstash.  
what can I do?

here it is my config.

```
input{
    file{
		path => "C:\SimDvcLk\tempLog.csv"
    	start_position => "beginning"
		sincedb_path => "/dev/null"
    }
}
filter{
    date{
	    match => ["Time", "yyyy-MM-dd HH:mm:ss"]
    	target => "@timestamp"
	}

	csv {
    	separator => ","
		columns => ["Time", "ControlMode","ActTemp","SetTemp","Output"]
	}

	mutate{convert=>["ControlMode","string"]}
	mutate{convert=>["ActTemp","float"]}
	mutate{convert=>["SetTemp","float"]}
	mutate{convert=>["Output","float"]}

}
output {
	elasticsearch {
    	hosts => "http://localhost:9200"
	    index => "templog"
	}
    stdout {}
}
```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 16, 2018, 4:22am UTC](https://discuss.elastic.co/t/csv-data-upload-problem/156956/2 "2018-11-16T04:22:53Z")

</div>

> [@f24b83985b804d90965e](#):
>
> sincedb\_path =\> "/dev/null"

Should be just `nul` on Windows.

> [@f24b83985b804d90965e](#):
>
> but @timestamp logging what I execute logstash.  
> what can I do?

Logstash executes things sequentially, so your `date` filter needs to go after the CSV one, otherwise it has no idea where the `Time` field is.

---

<div class="post-metadata">

**Author:** ![f24b83985b804d90965e](https://avatars.discourse-cdn.com/v4/letter/f/46a35a/32.png) [@f24b83985b804d90965e](https://discuss.elastic.co/u/f24b83985b804d90965e)\
**Post date:** [November 16, 2018, 5:17am UTC](https://discuss.elastic.co/t/csv-data-upload-problem/156956/3 "2018-11-16T05:17:43Z")

</div>

Thanks for the reply.  
but, still I have problem. ;(  
first, I upgrade to v6.5.0

1. CSV data upload  
sincedb\_path =\> "nul" is not occur error anymore. great. 🙂  
but, Logstash do not upload .CSV data.  

2. replace @timestamp  
I change .CSV file time stamp as ISO8601 format.  
then change config file like Logstash getting start video.  
also, not working at v6.2.2  
still, it is not working like video.  

please help.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 16, 2018, 7:06am UTC](https://discuss.elastic.co/t/csv-data-upload-problem/156956/4 "2018-11-16T07:06:45Z")

</div>

If you can show us a sample log line it may be easier to help find why it is not working.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 16, 2018, 7:26am UTC](https://discuss.elastic.co/t/csv-data-upload-problem/156956/5 "2018-11-16T07:26:42Z")

</div>

Also, please don't post pictures of text, they are difficult to read and some people may not be even able to see them.

---

<div class="post-metadata">

**Author:** ![f24b83985b804d90965e](https://avatars.discourse-cdn.com/v4/letter/f/46a35a/32.png) [@f24b83985b804d90965e](https://discuss.elastic.co/u/f24b83985b804d90965e)\
**Post date:** [November 16, 2018, 7:58am UTC](https://discuss.elastic.co/t/csv-data-upload-problem/156956/6 "2018-11-16T07:58:23Z")

</div>

sorry about picture. my bad.

sample csv format is like this.

```
  Time,ControlMode,ActTemp,SetTemp,Output
  2018-11-15T02:27:57.965Z,idle,25,50,0
  2018-11-15T02:27:57.967Z,tuning,25.2,50,100

```

rest of format is same.

and config file is

```
input{
	file{
		path => "C:\SimDvcLk\tempLog.csv"
		start_position => "beginning"
		sincedb_path => "nul"
	}
}
filter{
	csv {
		separator => ","
		columns => ["Time","ControlMode","ActTemp","SetTemp","Output"]
	}

	mutate{convert=>["Time","string"]}
	mutate{convert=>["ControlMode","string"]}
	mutate{convert=>["ActTemp","float"]}
	mutate{convert=>["SetTemp","float"]}
	mutate{convert=>["Output","float"]}
}
output {
	elasticsearch {
		hosts => "http://localhost:9200"
		index => "templog"
	}
stdout { codec => "rubydebug" }
}

```

I was remove date for now.  
thanks for helping me.

---

<div class="post-metadata">

**Author:** ![balumurari1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/balumurari1/32/39203_2.png) [@balumurari1](https://discuss.elastic.co/u/balumurari1)\
**Post date:** [November 20, 2018, 5:56am UTC](https://discuss.elastic.co/t/csv-data-upload-problem/156956/7 "2018-11-20T05:56:22Z")

</div>

hello,

if you were using windows machine make sure to change the path from

path =\> "C:\SimDvcLk\tempLog.csv"

to

path =\> "C:/SimDvcLk/tempLog.csv"

Hope this helps you,

Regards,  
Balu

---

<div class="post-metadata">

**Author:** ![f24b83985b804d90965e](https://avatars.discourse-cdn.com/v4/letter/f/46a35a/32.png) [@f24b83985b804d90965e](https://discuss.elastic.co/u/f24b83985b804d90965e)\
**Post date:** [November 21, 2018, 12:36am UTC](https://discuss.elastic.co/t/csv-data-upload-problem/156956/8 "2018-11-21T00:36:44Z")

</div>

still same. but don't worry.  
I use v6.5.0 except logstash.  
logstash v6.2.2 can upload to v6.5.0 elastic search.  
thank you.

---

<div class="post-metadata">

**Author:** ![balumurari1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/balumurari1/32/39203_2.png) [@balumurari1](https://discuss.elastic.co/u/balumurari1)\
**Post date:** [November 21, 2018, 5:07am UTC](https://discuss.elastic.co/t/csv-data-upload-problem/156956/9 "2018-11-21T05:07:51Z")

</div>

check this sample example for csv upload which is working fine,

> [@Load csv file in logstash](https://discuss.elastic.co/t/load-csv-file-in-logstash/151319/3):
>
> Thanks for reply,hope this works fine, input { file { path =\> "D:/Balu/ELK-stack/csvfile.csv" start\_position =\> "beginning" sincedb\_path =\> "NUL" } } filter { csv { separator =\> "," columns =\> ["open","high","low","close","volume"] } } output { elasticsearch { hosts =\> "[http://localhost:9200](http://localhost:9200)" index =\> "wallet-address-index" } stdout { codec =\> rubydebug } }

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 19, 2018, 5:07am UTC](https://discuss.elastic.co/t/csv-data-upload-problem/156956/10 "2018-12-19T05:07:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.