# Index not getting created in elasticsearch **URL:** \ **Category:** Logstash\ **Created:** [April 26, 2018, 7:14pm UTC](https://discuss.elastic.co/t/index-not-getting-created-in-elasticsearch/129737 "2018-04-26T19:14:44Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Maheshmadabul](https://avatars.discourse-cdn.com/v4/letter/m/34f0e0/32.png) [@Maheshmadabul](https://discuss.elastic.co/u/Maheshmadabul)\ **Post date:** [April 26, 2018, 7:14pm UTC](https://discuss.elastic.co/t/index-not-getting-created-in-elasticsearch/129737/1 "2018-04-26T19:14:44Z")
When i start the logstash with my conf file the following log getting printed and no index are getting created in the elastisearch Logstash log Sending Logstash's logs to E:/ELK/logstash-6.2.3/logs which is now configured via log4j2.properties [2018-04-27T00:36:49,496][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"E:/ELK/logstash-6.2.3/modules/fb\_apache/configuration"} [2018-04-27T00:36:49,527][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"E:/ELK/logstash-6.2.3/modules/netflow/configuration"} [2018-04-27T00:36:49,824][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified [2018-04-27T00:36:50,652][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.2.3"} [2018-04-27T00:36:51,558][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600} [2018-04-27T00:37:03,146][INFO][logstash.pipeline] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>4, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50} [2018-04-27T00:37:03,772][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>[], :added=\>[[http://localhost:9200/](http://localhost:9200/)]}} [2018-04-27T00:37:03,787][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://localhost:9200/](http://localhost:9200/), :path=\>"/"} [2018-04-27T00:37:04,084][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://localhost:9200/](http://localhost:9200/)"} [2018-04-27T00:37:04,178][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>6} [2018-04-27T00:37:04,178][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>6} [2018-04-27T00:37:04,209][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=\>nil} [2018-04-27T00:37:04,240][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"template"=\>"logstash-_", "version"=\>60001, "settings"=\>{"index.refresh\_interval"=\>"5s"}, "mappings"=\>{"default"=\>{"dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword", "ignore\_above"=\>256}}}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date"}, "@version"=\>{"type"=\>"keyword"}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"type"=\>"half\_float"}}}}}}}} [2018-04-27T00:37:04,318][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[//localhost:9200](https://localhost:9200)"]} [2018-04-27T00:37:04,350][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>[], :added=\>[[http://localhost:9200/](http://localhost:9200/)]}} [2018-04-27T00:37:04,350][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://localhost:9200/](http://localhost:9200/), :path=\>"/"} [2018-04-27T00:37:04,350][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://localhost:9200/](http://localhost:9200/)"} [2018-04-27T00:37:04,381][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>6} [2018-04-27T00:37:04,381][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>6} [2018-04-27T00:37:04,381][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=\>nil} [2018-04-27T00:37:04,397][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"template"=\>"logstash-_", "version"=\>60001, "settings"=\>{"index.refresh\_interval"=\>"5s"}, "mappings"=\>{"default"=\>{"dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword", "ignore\_above"=\>256}}}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date"}, "@version"=\>{"type"=\>"keyword"}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"type"=\>"half\_float"}}}}}}}} [2018-04-27T00:37:04,428][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[//localhost:9200](https://localhost:9200)"]} [2018-04-27T00:37:06,162][INFO][logstash.pipeline] Pipeline started succesfully {:pipeline\_id=\>"main", :thread=\>"#\"} [2018-04-27T00:37:06,381][INFO][logstash.agent] Pipelines running {:count=\>1, :pipelines=\>["main"]} My conf file: input { file { path =\> "C:\Users\Windows\Desktop\Prod\web1.log" type =\> "web1\_accesslogs" } ``` file { path => "C:\Users\Windows\Desktop\Prod\web2.log" type => "web2_accesslogs" } ``` } filter { if [type] == "web1\_accesslogs" { grok{ match =\> { "message" =\> "%{IPORHOST:clientip} %{USER:ident} %{USER:auth} [%{HTTPDATE:logTime}] "%{WORD:method} %{URIPATHPARAM:request}\ %{NOTSPACE:httpV}" %{NUMBER:status} (?:-|%{NUMBER:bytes}) %{QS:referrer} %{QS:agent} %{NUMBER:responseTime} %{NUMBER:responseTime}" } } } if [type] == "web2\_accesslogs" { grok { match =\> { "message" =\> "%{IPORHOST:clientip} %{USER:ident} %{USER:auth} [%{HTTPDATE:logTime}] "%{WORD:method} %{URIPATHPARAM:request}\ %{NOTSPACE:httpV}" %{NUMBER:status} (?:-|%{NUMBER:bytes}) %{QS:referrer} %{QS:agent} %{NUMBER:responseTime} %{NUMBER:responseTime}" } } } ``` mutate { convert => { "bytes" => "integer"} convert => { "status" => "integer"} convert => { "responseTime" => "integer"} } useragent { source => "agent" target => "useragent" } date { match => ["logTime", "dd/MMM/YYYY:HH:mm:ss Z"] locale => en } ``` } output { if [type] == "web1\_accesslogs" { elasticsearch { hosts =\> ["localhost:9200"] index =\> "web1\_logs" } } if [type] == "web2\_accesslogs" { elasticsearch { hosts =\> ["localhost:9200"] index =\> "web2\_logs" } } ``` stdout { codec => rubydebug } ``` } ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 26, 2018, 7:57pm UTC](https://discuss.elastic.co/t/index-not-getting-created-in-elasticsearch/129737/2 "2018-04-26T19:57:07Z")
How do you know Logstash is even reading anything from your files? Have you read the file input documentation, paying special attention to the `start_position` option and everything that's said about sincedb? ---
**Author:** ![Maheshmadabul](https://avatars.discourse-cdn.com/v4/letter/m/34f0e0/32.png) [@Maheshmadabul](https://discuss.elastic.co/u/Maheshmadabul)\ **Post date:** [April 27, 2018, 5:54am UTC](https://discuss.elastic.co/t/index-not-getting-created-in-elasticsearch/129737/3 "2018-04-27T05:54:02Z")
Added the start\_position but still its not reading from files. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 27, 2018, 6:23am UTC](https://discuss.elastic.co/t/index-not-getting-created-in-elasticsearch/129737/4 "2018-04-27T06:23:07Z")
`start_position` only makes a difference the first time a file is seen. In your case Logstash has already decided upon a position in the file. This is an extremely common problem that people have. Please look into past threads for elaborations. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 25, 2018, 6:23am UTC](https://discuss.elastic.co/t/index-not-getting-created-in-elasticsearch/129737/5 "2018-05-25T06:23:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.