# Can't input in Elasticsearch **URL:** \ **Category:** Logstash\ **Created:** [June 6, 2016, 3:20pm UTC](https://discuss.elastic.co/t/cant-input-in-elasticsearch/51972 "2016-06-06T15:20:12Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Kevin\_Csuka](https://avatars.discourse-cdn.com/v4/letter/k/7feea3/32.png) [@Kevin\_Csuka](https://discuss.elastic.co/u/Kevin_Csuka)\ **Post date:** [June 6, 2016, 3:20pm UTC](https://discuss.elastic.co/t/cant-input-in-elasticsearch/51972/1 "2016-06-06T15:20:12Z")
So I'm an all goner here. Got no clue where my mistake is. Here's the deal: I'm trying to input data from a csv file in Elasticsearch 2.3.2. The input, filter and output is done by Logstash 2.3.2. The .csv file: ```auto timeStamp,elapsed,label,responseCode,responseMessage,threadName,success,grpThreads,allThreads,Latency,Hostname,Connect 1464772797973,1082,http://test4,200,OK,ThreadGroup 1-1,true,2,2,318,DESKTOP-NS,166 1464772798501,1011,http://test4.com,200,OK,ThreadGroup 1-2,true,3,3,185,DESKTOP-NS,94 1464772799061,889,http://test4.com,200,OK,ThreadGroup 1-1,true,3,3,87,DESKTOP-NS,0 ``` The test.conf of Logstash: [code] input { file { type =\> "csv" path =\> ["C:\result\results.csv"] start\_position =\> "beginning" } } filter { if [elapsed] == "label" { drop { } } ``` csv { columns => ["@jmeter_timestamp", "elapsed", "label", "responseCode", "responseMessage", "threadName", "success", "grpThreads", "allThreads", "Latency", "Hostname", "Connect"] ``` separator =\> "," } date { match =\> ["@jmeter\_timestamp", "UNIX\_MS"] target =\> "@timestamp" timezone =\> "Europe/Amsterdam" remove\_field =\> ["jmeter\_timestamp"] } } output { elasticsearch { hosts =\> ["[http://192.168.43.51:9200](http://192.168.43.51:9200)"] index =\> "jmeter-%{+YYYY.MM.dd}" } } [/code] My Index mapping: ```auto { "template": "jmeter-*", "settings": { "number_of_shards": 1, "number_of_replicas": 0, "index.refresh_interval": "10s" }, "mappings": { "logs": { "properties": { "@timestamp": { "type": "date", "format": "yyyyMMdd'T'HHmmss.SSSZ" }, "elapsed": { "type": "integer" }, "label": { "type": "string" }, "responseCode": { "type": "integer" }, "responseMessage": { "type": "string" }, "threadName": { "type": "string" }, "success": { "type": "boolean" }, "grpThreads": { "type": "long" }, "allThreads": { "type": "long" }, "Latency": { "type": "long" }, "ErrorCount": { "type": "long" }, "Hostname": { "type": "string" }, "Connect": { "type": "long" } } } } } ``` Kibana is showing no data from any timezone: [http://i.imgur.com/OMsJTXz.png](http://i.imgur.com/OMsJTXz.png) The output of stdout of Logstash is showing the following: [http://i.imgur.com/yez54yD.png](http://i.imgur.com/yez54yD.png) I tried to remove, and re-add the index with no success. Can anyone pin-point me to my mistake. Thanks ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [June 6, 2016, 10:25pm UTC](https://discuss.elastic.co/t/cant-input-in-elasticsearch/51972/2 "2016-06-06T22:25:59Z")
Can you connect to your ES host from LS? eg with telnet ---
**Author:** ![Kevin\_Csuka](https://avatars.discourse-cdn.com/v4/letter/k/7feea3/32.png) [@Kevin\_Csuka](https://discuss.elastic.co/u/Kevin_Csuka)\ **Post date:** [June 7, 2016, 9:50am UTC](https://discuss.elastic.co/t/cant-input-in-elasticsearch/51972/3 "2016-06-07T09:50:53Z")
Yeah, connection-wise everything is good. See the following: [http://imgur.com/icRo34t](http://imgur.com/icRo34t) ---
**Author:** ![Kevin\_Csuka](https://avatars.discourse-cdn.com/v4/letter/k/7feea3/32.png) [@Kevin\_Csuka](https://discuss.elastic.co/u/Kevin_Csuka)\ **Post date:** [June 8, 2016, 1:50pm UTC](https://discuss.elastic.co/t/cant-input-in-elasticsearch/51972/4 "2016-06-08T13:50:34Z")
Anyone? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 8, 2016, 6:35pm UTC](https://discuss.elastic.co/t/cant-input-in-elasticsearch/51972/5 "2016-06-08T18:35:34Z")
Increasing Logstash's log level by starting it with `--verbose` or even `--debug` could give clues. ---
**Author:** ![Kevin\_Csuka](https://avatars.discourse-cdn.com/v4/letter/k/7feea3/32.png) [@Kevin\_Csuka](https://discuss.elastic.co/u/Kevin_Csuka)\ **Post date:** [June 9, 2016, 8:28am UTC](https://discuss.elastic.co/t/cant-input-in-elasticsearch/51972/6 "2016-06-09T08:28:23Z")
> [@magnusbaeck](#): > > g level by starti Both the messages from, --debug and --verbose, didn't give me any clues. Here's the result: [http://imgur.com/W7tbIvM](http://imgur.com/W7tbIvM) (Click on image for full screen) I tried the following: 1. Deleted the 'jmeter-\*' index pattern in Elasticsearch. 2. Configured Logstash to input the template 'jmeter.json' in Elasticsearch. 3. Start Logstash again, and I see the 'jmeter-' index pattern added in kopf. 4. In Kibana I add the index pattern again. 5. At the Discover tab, there is no data available, from 15 minutes to 5 years ago * * * I think the problem lays in the UNIX\_MS timestamp. The result.csv has a UNIX\_MS timestamp, e.g. `1464772797973` My JSON index pattern file is set like this: ` "properties": { "@timestamp": { "type": "date", "format": "yyyyMMdd'T'HHmmss.SSSZ" },` And my config in Logstash is setup like this: ` csv { columns => ["@jmeter_timestamp", ..."] separator => "," } date { match => ["@jmeter_timestamp", "UNIX_MS"] target => "@timestamp" timezone => "Europe/Amsterdam" remove_field => ["jmeter_timestamp"] } } ` Are these settings correct? Note: the full/original files are in the first post ---
**Author:** ![Kevin\_Csuka](https://avatars.discourse-cdn.com/v4/letter/k/7feea3/32.png) [@Kevin\_Csuka](https://discuss.elastic.co/u/Kevin_Csuka)\ **Post date:** [June 13, 2016, 3:30pm UTC](https://discuss.elastic.co/t/cant-input-in-elasticsearch/51972/7 "2016-06-13T15:30:16Z")
Allright, last try here. I don't want to open a second thread. I'm trying to input two values in Elasticsearch through Logstash. The first value is a UNIX\_MS timestamp, the second is simply an integer: ```auto ``` My test.json which Logstash sends to Elasticsearch: ` { "template": "alles-*", "settings": { "number_of_shards": 1, "number_of_replicas": 0, "index.refresh_interval": "10s" }, "mappings": { "logs": { "properties": { "@timestamp": { "type": "date", "format": "yyyyMMdd'T'HHmmss.SSS" }, "elapsed": { "type": "integer" } } } } } ` And finally my test.conf file ` input { file { type => "csv" path => ["C:/result2/results.jtl"] start_position => "beginning" } } filter { csv { columns => ["@jmeter_timestamp", "elapsed"] separator => "," } date { match => ["@jmeter_timestamp", "UNIX_MS"] target => "@timestamp" timezone => "Europe/Amsterdam" remove_field => ["jmeter_timestamp"] } } output { elasticsearch { template => "c:/result2/jmeter.json" template_name => "alles" hosts => ["192.168.43.51:9200"] index => "alles-%{+YYYY.MM.dd}" } } ` When I run: ` logstash -f test.conf -- debug` or ` --verbose ` it shows no errors. Still, no data is added in Elasticsearch. But the JSON template is added. [http://imgur.com/ONlpl9i](http://imgur.com/ONlpl9i) ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 13, 2016, 5:31pm UTC](https://discuss.elastic.co/t/cant-input-in-elasticsearch/51972/8 "2016-06-13T17:31:54Z")
Logstash is probably tailing the input file since it already processed the data once. Set the file input's `sincedb_path` option to "nul" or use the stdin input and pipe the data to Logstash. If you stick to the file input make sure you also adjust `ignore_older` if the input file is older than 24 hours. This topic here a couple of times a week so there's plenty more in the archives (and sincedb is described in the file input documentation). There are a couple of weird things in your date filter but let's deal with that later. ---
**Author:** ![Kevin\_Csuka](https://avatars.discourse-cdn.com/v4/letter/k/7feea3/32.png) [@Kevin\_Csuka](https://discuss.elastic.co/u/Kevin_Csuka)\ **Post date:** [June 14, 2016, 7:41am UTC](https://discuss.elastic.co/t/cant-input-in-elasticsearch/51972/9 "2016-06-14T07:41:58Z")
> [@magnusbaeck](#): > > re are a couple of weird things in your date filter but let's deal wit This is so weird... I adjusted my Logstash config to: ` input { file { sincedb_path => "NUL" ignore_older => 0 type => "csv" path => ["C:/result2/results.jtl"] start_position => "beginning" } } ` And now, it shows data in Elasticsearch. I guess Logstash only took data from cache. Thanks @magnusbaeck * * * ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:53am UTC](https://discuss.elastic.co/t/cant-input-in-elasticsearch/51972/10 "2017-07-06T04:53:06Z")
Closed on July 6, 2017, 4:53am UTC