# Setting and testing logstash and elastic search intsallation
**URL:** https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971
**Category:** Logstash
**Created:** [April 29, 2018, 5:22pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971 "2018-04-29T17:22:45Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@rbradley](https://discuss.elastic.co/u/rbradley)
#### Post date: [April 29, 2018, 5:22pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971/1 "2018-04-29T17:22:45Z")
Hi,
I am trying to load a csv file into logtsash and output to elastic search.
i have read some of the topics and my error is similar to them.
I dont think index has been created.
what should i do?
---
### Author:  [@atira](https://discuss.elastic.co/u/atira)
#### Post date: [April 29, 2018, 9:35pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971/2 "2018-04-29T21:35:16Z")
I don't see any error messages on the screenshot.
> [@rbradley](#):
>
> I dont think index has been created.
? Did you check ES if it had the index? Did you check ES's logs?
How does your Logstash config look like? Are you actually feeding Logstash? Index will be created once there is data loaded in it.
---
### Author:  [@rbradley](https://discuss.elastic.co/u/rbradley)
#### Post date: [April 30, 2018, 1:10pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971/3 "2018-04-30T13:10:59Z")
I am trying to load data from a CSV file
data is like this:
11-Aug,123,eXXX,200,bXXX,404\_TRN\_IDENTIFIER\_NOT\_FOUND,XX-number (identifier) provided not found
My colleagure was able to load it- his stack is on mac in a docker. i am on windows and we have restrictions on our C drive. I have elastic search loaded as a service and is in c: program files. logstash and Kibana were on my downloads folder- now, i am going to try and move them to my documents and then retry.
here is my log:
yes, index hasnt been created.
---
### Author:  [@rbradley](https://discuss.elastic.co/u/rbradley)
#### Post date: [April 30, 2018, 4:12pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971/4 "2018-04-30T16:12:45Z")
when i ran in debug mode: I get the following error:
[2018-04-30T17:11:07,265][DEBUG][logstash.inputs.file] \_globbed\_files: C:\Users\rbradley\Documents\logstash-6.2.4\logstash-6.2.4\informatica.log: glob is: []
[2018-04-30T17:11:07,267][DEBUG][logstash.inputs.file] \_globbed\_files: C:\Users\rbradley\Documents\logstash-6.2.4\logstash-6.2.4\informatica.log: glob is: ["C:\Users\rbradley\Documents\logstash-6.2.4\logstash-6.2.4\informatica.log"] because glob did not work
---
### Author:  [@rbradley](https://discuss.elastic.co/u/rbradley)
#### Post date: [April 30, 2018, 4:25pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971/5 "2018-04-30T16:25:19Z")
here is where it says skipping files:
[2018-04-30T17:10:41,534][DEBUG][logstash.config.source.local.configpathloader] Skipping the following files while reading config since they don't match the specified glob pattern {:files=\>["C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/CONTRIBUTORS", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/Gemfile", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/Gemfile.lock", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/LICENSE", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/NOTICE.TXT", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/bin", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/config", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/data", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/lib", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/logs", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/logstash-core", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/logstash-core-plugin-api", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/modules", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/sample-errors-test.csv", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/tools", "C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/vendor"]}
[2018-04-30T17:10:41,561][DEBUG][logstash.api.service] [api-service] start
[2018-04-30T17:10:41,567][DEBUG][logstash.config.source.local.configpathloader] Reading config file {:config\_file=\>"C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/logstash-simple.conf"}
---
### Author:  [@rbradley](https://discuss.elastic.co/u/rbradley)
#### Post date: [April 30, 2018, 4:26pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971/6 "2018-04-30T16:26:33Z")
trying to access a template:
[2018-04-30T17:10:50,436][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=\>nil}
[2018-04-30T17:10:50,519][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"template"=\>"logstash-_", "version"=\>60001, "settings"=\>{"index.refresh\_interval"=\>"5s"}, "mappings"=\>{"default"=\>{"dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword", "ignore\_above"=\>256}}}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date"}, "@version"=\>{"type"=\>"keyword"}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"type"=\>"half\_float"}}}}}}}}
[2018-04-30T17:10:50,654][DEBUG][logstash.outputs.elasticsearch] Found existing Elasticsearch template. Skipping template management {:name=\>"logstash"}
[2018-04-30T17:10:50,682][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[//localhost:9200](https://localhost:9200)"]}
[2018-04-30T17:10:51,103][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=\>"ParNew"}
---
### Author:  [@rbradley](https://discuss.elastic.co/u/rbradley)
#### Post date: [April 30, 2018, 4:27pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971/7 "2018-04-30T16:27:59Z")
glob error
[2018-04-30T17:10:52,804][DEBUG][logstash.inputs.file] \_globbed\_files: C:\Users\rbradley\Documents\logstash-6.2.4\logstash-6.2.4\informatica.log: glob is: []
[2018-04-30T17:10:52,815][DEBUG][logstash.inputs.file] \_globbed\_files: C:\Users\rbradley\Documents\logstash-6.2.4\logstash-6.2.4\informatica.log: glob is: ["C:\Users\rbradley\Documents\logstash-6.2.4\logstash-6.2.4\informatica.log"] because glob did not work
[2018-04-30T17:10:52,847][DEBUG][logstash.inputs.file] \_discover\_file: C:\Users\rbradley\Documents\logstash-6.2.4\logstash-6.2.4\informatica.log: new: C:\Users\rbradley\Documents\logstash-6.2.4\logstash-6.2.4\informatica.log (exclude is [])
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 30, 2018, 5:08pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971/8 "2018-04-30T17:08:29Z")
> [2018-04-30T17:10:52,804][DEBUG][logstash.inputs.file] \_globbed\_files: C:\Users\rbradley\Documents\logstash-6.2.4\logstash-6.2.4\informatica.log: glob is:
Either the filename pattern is wrong or the user that Logstash runs as doesn't have sufficient permissions. Do you really have two nested logstash-6.2.4 directories?
---
### Author:  [@rbradley](https://discuss.elastic.co/u/rbradley)
#### Post date: [April 30, 2018, 7:18pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971/9 "2018-04-30T19:18:47Z")
yes i do..yes, a part of me thinks it may be access issue. what permissions you should have to be able to run logstash?
I found your reply on someone else' question- is this still valid? The user directory is mode drwxrw-rw- (766) but it's important that directory is executable (drwxrwxrwx, 777). It does however not need to be writable so what you're really looking for is (drwxr-xr-x, 755). Same thing with the Desktop directory, it doesn't have to be writable to anyone but you.
should i get my helpdesk to check my access to the directory properly?
The file is good as my colleague is able to execute on his docker on mac
---
### Author:  [@rbradley](https://discuss.elastic.co/u/rbradley)
#### Post date: [April 30, 2018, 8:02pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971/10 "2018-04-30T20:02:45Z")
okay, I have corrected my glob error by changing slash to path =\> "C:/ instead of path =\> "C:\
now in the debug mode- it seems to be
a) has a timeut exception
[2018-04-30T21:10:25,967][DEBUG][logstash.instrument.periodicpoller.persistentqueue] Timeout exception {:poller=\>#\, @listeners=\\>, @id\_path="c:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/data/uuid", @running=\<#Concurrent::AtomicBoolean:0xfb0 value:true\>, @periodic\_pollers=#\, @structured\_lookup\_mutex=#Mutex:0xbc294cc, @fast\_lookup=#\\>\>\>, @periodic\_pollers=[#\, @structured\_lookup\_mutex=#Mutex:0xbc294cc, @fast\_lookup=#\\>\>\>, @task=#\=\>:update}\>, @timeout\_interval=120.0, @running=\<#Concurrent::AtomicBoolean:0xfbc value:true\>, @StoppedEvent=#\
### Author:  [@rbradley](https://discuss.elastic.co/u/rbradley)
#### Post date: [April 30, 2018, 8:03pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971/11 "2018-04-30T20:03:01Z")
and in normal mode, it is:
c:\Users\rbradley\Documents\logstash-6.2.4\logstash-6.2.4\bin\>logstash -f c:\Users\rbradley\Documents\logstash-6.2.4\logstash-6.2.4\logstash-simple.conf
Sending Logstash's logs to c:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/logs which is now configured via log4j2.properties
[2018-04-30T20:52:04,185][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"c:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/modules/fb\_apache/configuration"}
[2018-04-30T20:52:04,211][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"c:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/modules/netflow/configuration"}
[2018-04-30T20:52:04,625][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-04-30T20:52:05,446][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.2.4"}
[2018-04-30T20:52:06,855][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600}
[2018-04-30T20:52:13,544][INFO][logstash.pipeline] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>4, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50}
[2018-04-30T20:52:14,220][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>[], :added=\>[[http://localhost:9200/](http://localhost:9200/)]}}
[2018-04-30T20:52:14,244][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://localhost:9200/](http://localhost:9200/), :path=\>"/"}
[2018-04-30T20:52:14,504][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://localhost:9200/](http://localhost:9200/)"}
[2018-04-30T20:52:14,616][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>6}
[2018-04-30T20:52:14,626][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>6}
[2018-04-30T20:52:14,661][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=\>nil}
[2018-04-30T20:52:14,695][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"template"=\>"logstash-_", "version"=\>60001, "settings"=\>{"index.refresh\_interval"=\>"5s"}, "mappings"=\>{"default"=\>{"dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword", "ignore\_above"=\>256}}}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date"}, "@version"=\>{"type"=\>"keyword"}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"type"=\>"half\_float"}}}}}}}}
[2018-04-30T20:52:14,782][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[//localhost:9200](https://localhost:9200)"]}
[2018-04-30T20:52:16,173][INFO][logstash.pipeline] Pipeline started successfully {:pipeline\_id=\>"main", :thread=\>"#\"}
[2018-04-30T20:52:16,370][INFO][logstash.agent] Pipelines running {:count=\>1, :pipelines=\>["main"]}
---
### Author:  [@rbradley](https://discuss.elastic.co/u/rbradley)
#### Post date: [April 30, 2018, 8:31pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971/12 "2018-04-30T20:31:33Z")
b) b) going in a loop:
[2018-04-30T20:34:06,002][INFO][logstash.pipeline] Pipeline started successfully {:pipeline\_id=\>"main", :thread=\>"#\"}
[2018-04-30T20:34:06,224][INFO][logstash.agent] Pipelines running {:count=\>1, :pipelines=\>["main"]}
[2018-04-30T20:34:06,307][DEBUG][logstash.inputs.file] \_globbed\_files: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: glob is: ["C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log"]
[2018-04-30T20:34:06,318][DEBUG][logstash.inputs.file] \_discover\_file: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: new: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log (exclude is [])
[2018-04-30T20:34:06,459][DEBUG][logstash.inputs.file] \_open\_file: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: opening
[2018-04-30T20:34:06,496][DEBUG][logstash.inputs.file] C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: initial create, no sincedb, seeking to end 6060
[2018-04-30T20:34:06,549][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:07,565][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:08,574][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:09,585][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:10,594][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:10,737][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=\>"ParNew"}
[2018-04-30T20:34:10,743][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=\>"ConcurrentMarkSweep"}
[2018-04-30T20:34:11,039][DEBUG][logstash.pipeline] Pushing flush onto pipeline {:pipeline\_id=\>"main", :thread=\>"#\"}
[2018-04-30T20:34:11,601][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:12,604][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:13,612][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:14,620][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:15,630][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:15,783][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=\>"ParNew"}
[2018-04-30T20:34:15,786][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=\>"ConcurrentMarkSweep"}
[2018-04-30T20:34:16,047][DEBUG][logstash.pipeline] Pushing flush onto pipeline {:pipeline\_id=\>"main", :thread=\>"#\"}
[2018-04-30T20:34:16,637][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:17,642][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:18,650][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:19,660][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:20,665][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:20,677][DEBUG][logstash.inputs.file] \_globbed\_files: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: glob is: ["C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log"]
[2018-04-30T20:34:20,801][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=\>"ParNew"}
[2018-04-30T20:34:20,803][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=\>"ConcurrentMarkSweep"}
[2018-04-30T20:34:21,049][DEBUG][logstash.pipeline] Pushing flush onto pipeline {:pipeline\_id=\>"main", :thread=\>"#\"}
[2018-04-30T20:34:21,686][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:22,690][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:23,695][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:24,703][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
[2018-04-30T20:34:25,709][DEBUG][logstash.inputs.file] each: file grew: C:/Users/rbradley/Documents/logstash-6.2.4/logstash-6.2.4/informatica.log: old size 0, new size 6060
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 28, 2018, 8:31pm UTC](https://discuss.elastic.co/t/setting-and-testing-logstash-and-elastic-search-intsallation/129971/13 "2018-05-28T20:31:39Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.