# Cannot get indexes made by logstash

**URL:** <https://discuss.elastic.co/t/cannot-get-indexes-made-by-logstash/83163>\
**Category:** Logstash\
**Created:** [April 21, 2017, 7:56am UTC](https://discuss.elastic.co/t/cannot-get-indexes-made-by-logstash/83163 "2017-04-21T07:56:32Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![sLuvpreet33](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sluvpreet33/32/50626_2.png) [@sLuvpreet33](https://discuss.elastic.co/u/sLuvpreet33)\
**Post date:** [April 21, 2017, 7:56am UTC](https://discuss.elastic.co/t/cannot-get-indexes-made-by-logstash/83163/1 "2017-04-21T07:56:32Z")

</div>

I think my logstash is not working,

Here is the configuration of logstash,

```
input {
  file {
     type => nginx
     path => "/var/log/nginx/kibana_access.log"
    }
 }
output {
  elasticsearch {
    hosts => ["localhost:9200"]
    manage_template => false
    index => "%{type}-index"
    document_type => "[type]"
  }
}

```

But I cannot see any kind of new index made in elasticsearch made by this,  
This is the command I use to get all indexes,

`curl -XGET 'localhost:9200/_cat/indices?v&pretty'`

Why so ? Both services are up and running.

---

<div class="post-metadata">

**Author:** ![pablosan](https://avatars.discourse-cdn.com/v4/letter/p/e19adc/32.png) [@pablosan](https://discuss.elastic.co/u/pablosan)\
**Post date:** [April 21, 2017, 7:58am UTC](https://discuss.elastic.co/t/cannot-get-indexes-made-by-logstash/83163/2 "2017-04-21T07:58:06Z")

</div>

I would first check the Logstash logs to see what's going on.

---

<div class="post-metadata">

**Author:** ![sLuvpreet33](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sluvpreet33/32/50626_2.png) [@sLuvpreet33](https://discuss.elastic.co/u/sLuvpreet33)\
**Post date:** [April 21, 2017, 8:44am UTC](https://discuss.elastic.co/t/cannot-get-indexes-made-by-logstash/83163/3 "2017-04-21T08:44:43Z")

</div>

@pablosan  
Looking at logs, I think the logstash is doing fine, Here have a look,  
I have shortened them a bit, as the body was exceeding 7000 characters,

```
[2017-04-21T13:27:30,211][DEBUG][logstash.runner] 
The following is the merged configuration
[2017-04-21T13:27:30,211][DEBUG][logstash.runner] 
input {
  file {
    type => nginx
    path => "/var/log/nginx/kibana_access.log"
   }
 }

output {
elasticsearch {
   hosts => ["localhost:9200"]
   manage_template => false
   index => "%{type}-index"
   document_type => "[type]"
 }
}

 [2017-04-21T13:27:30,235][DEBUG][logstash.basepipeline] Compiled pipeline code {:code=>" @inputs = []\n @filters = []\n @outputs = []\n @periodic_flushers = []\n @shutdown_flushers = []\n @generated_objects = {}\n\n @generated_objects[:input_file_1] = plugin(\"input\", \"file\", LogStash::Util.hash_merge_many({ \"type\" => (\"nginx\") }, { \"path\" => (\"/var/log/nginx/kibana_access.log\") }))\n\n @inputs << @generated_objects[:input_file_1]\n\n @generated_objects[:output_elasticsearch_2] = plugin(\"output\", \"elasticsearch\", LogStash::Util.hash_merge_many({ \"hosts\" => [(\"localhost:9200\")] }, { \"manage_template\" => (\"false\") }, { \"index\" => (\"%{type}-index\") }, { \"document_type\" => (\"[type]\") }))\n\n @outputs << @generated_objects[:output_elasticsearch_2]\n\n define_singleton_method :filter_func do |event|\n events = [event]\n @logger.debug? && @logger.debug(\"filter received\", \"event\" => event.to_hash)\n events\n end\n define_singleton_method :output_func do |event|\n targeted_outputs = []\n @logger.debug? && @logger.debug(\"output received\", \"event\" => event.to_hash)\n targeted_outputs << @generated_objects[:output_elasticsearch_2]\n \n targeted_outputs\n end"}
[2017-04-21T13:27:30,261][DEBUG][logstash.plugins.registry] On demand adding plugin to the registry {:name=>"file", :type=>"input", :class=>LogStash::Inputs::File}
[2017-04-21T13:27:30,270][DEBUG][logstash.plugins.registry] On demand adding plugin to the registry {:name=>"plain", :type=>"codec", :class=>LogStash::Codecs::Plain}
[2017-04-21T13:27:30,273][DEBUG][logstash.codecs.plain] config LogStash::Codecs::Plain/@id = "plain_61376031-3058-4698-969e-e611013fa608"
[2017-04-21T13:27:30,274][DEBUG][logstash.codecs.plain] config LogStash::Codecs::Plain/@enable_metric = true
[2017-04-21T13:27:30,274][DEBUG][logstash.codecs.plain] config LogStash::Codecs::Plain/@charset = "UTF-8"
[2017-04-21T13:27:30,275][DEBUG][logstash.inputs.file] config LogStash::Inputs::File/@type = "nginx"
[2017-04-21T13:27:30,276][DEBUG][logstash.inputs.file] config LogStash::Inputs::File/@path = ["/var/log/nginx/kibana_access.log"]
   [2017-04-21T13:27:30,277][DEBUG][logstash.inputs.file] config LogStash::Inputs::File/@stat_interval = 1
[2017-04-21T13:27:30,277][DEBUG][logstash.inputs.file] config LogStash::Inputs::File/@discover_interval = 15
[2017-04-21T13:27:30,277][DEBUG][logstash.inputs.file] config LogStash::Inputs::File/@sincedb_write_interval = 15
[2017-04-21T13:27:30,279][DEBUG][logstash.inputs.file] config LogStash::Inputs::File/@start_position = "end"
[2017-04-21T13:27:30,279][DEBUG][logstash.inputs.file] config LogStash::Inputs::File/@delimiter = "\n"
[2017-04-21T13:27:30,279][DEBUG][logstash.inputs.file] config LogStash::Inputs::File/@close_older = 3600
[2017-04-21T13:27:30,484][DEBUG][logstash.plugins.registry] On demand adding plugin to the registry {:name=>"elasticsearch", :type=>"output", :class=>LogStash::Outputs::ElasticSearch}
[2017-04-21T13:27:30,495][DEBUG][logstash.codecs.plain] config LogStash::Codecs::Plain/@id = "plain_96e60119-2cc8-4dc5-9ba7-26d0feb37ed6"
[2017-04-21T13:27:30,496][DEBUG][logstash.codecs.plain] config LogStash::Codecs::Plain/@enable_metric = true
[2017-04-21T13:27:30,497][DEBUG][logstash.codecs.plain] config LogStash::Codecs::Plain/@charset = "UTF-8"
[2017-04-21T13:27:30,504][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@hosts = [//localhost:9200]
[2017-04-21T13:27:30,504][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@manage_template = false
[2017-04-21T13:27:30,504][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@index = "%{type}-index"
[2017-04-21T13:27:30,504][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@document_type = "[type]"
[2017-04-21T13:27:30,505][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@id = "fd36a650089d7d9fbc7505a2c84eaae8af276ba6-2"
[2017-04-21T13:27:30,505][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@enable_metric = true
[2017-04-21T13:27:30,505][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@codec = <LogStash::Codecs::Plain id=>"plain_96e60119-2cc8-4dc5-9ba7-26d0feb37ed6", enable_metric=>true, charset=>"UTF-8">
[2017-04-21T13:27:30,505][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@workers = 1
[2017-04-21T13:27:30,505][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@template_name = "logstash"
[2017-04-21T13:27:30,505][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@template_overwrite = false
[2017-04-21T13:27:30,506][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@parent = nil
[2017-04-21T13:27:30,506][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@idle_flush_time = 1
[2017-04-21T13:27:30,506][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@upsert = ""
[2017-04-21T13:27:30,506][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@doc_as_upsert = false
[2017-04-21T13:27:30,506][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@script = ""
[2017-04-21T13:27:30,506][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@script_type = "inline"
[2017-04-21T13:27:30,507][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@retry_on_conflict = 1
[2017-04-21T13:27:30,507][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@pipeline = nil
[2017-04-21T13:27:30,520][INFO][logstash.runner] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
```

---

<div class="post-metadata">

**Author:** ![pablosan](https://avatars.discourse-cdn.com/v4/letter/p/e19adc/32.png) [@pablosan](https://discuss.elastic.co/u/pablosan)\
**Post date:** [April 21, 2017, 8:53am UTC](https://discuss.elastic.co/t/cannot-get-indexes-made-by-logstash/83163/4 "2017-04-21T08:53:18Z")

</div>

You can use the stdout debug to see if there are any logs being processed, and probably set the log level to trace in logstash.yml to try to understand why the logs are not picked up.

```
stdout {
  codec => rubydebug
}

```

You were testing the config there, right?

> [@sLuvpreet33](#):
>
> [2017-04-21T13:27:30,520][INFO][logstash.runner] Using config.test\_and\_exit mode. Config Validation Result: OK. Exiting Logstash

One thing that could be happening is that the log files are old and logstash won't use them.

---

<div class="post-metadata">

**Author:** ![sLuvpreet33](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sluvpreet33/32/50626_2.png) [@sLuvpreet33](https://discuss.elastic.co/u/sLuvpreet33)\
**Post date:** [April 21, 2017, 9:07am UTC](https://discuss.elastic.co/t/cannot-get-indexes-made-by-logstash/83163/5 "2017-04-21T09:07:12Z")

</div>

@pablosan it gives this ,

```
[2017-04-21T14:35:50,474][FATAL][logstash.runner] The given configuration is invalid. Reason: Couldn't find any input plugin named 'stdout'. Are you sure this is correct? Trying to load the stdout input plugin resulted in this error: Problems loading the requested plugin named stdout of type input. Error: NameError NameError
```

---

<div class="post-metadata">

**Author:** ![pablosan](https://avatars.discourse-cdn.com/v4/letter/p/e19adc/32.png) [@pablosan](https://discuss.elastic.co/u/pablosan)\
**Post date:** [April 21, 2017, 9:09am UTC](https://discuss.elastic.co/t/cannot-get-indexes-made-by-logstash/83163/6 "2017-04-21T09:09:17Z")

</div>

Sorry I probably didn't explain where to put it, it goes in the output tag

[https://www.elastic.co/guide/en/logstash/current/plugins-outputs-stdout.html](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-stdout.html)

---

<div class="post-metadata">

**Author:** ![sLuvpreet33](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sluvpreet33/32/50626_2.png) [@sLuvpreet33](https://discuss.elastic.co/u/sLuvpreet33)\
**Post date:** [April 21, 2017, 9:18am UTC](https://discuss.elastic.co/t/cannot-get-indexes-made-by-logstash/83163/7 "2017-04-21T09:18:29Z")

</div>

One thing is that ,

```
[2017-04-21T14:47:03,712][INFO][logstash.runner] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

```

It is using config\_test\_and\_exit mode, I have not told it to do so. Can this be the problem ?

---

<div class="post-metadata">

**Author:** ![pablosan](https://avatars.discourse-cdn.com/v4/letter/p/e19adc/32.png) [@pablosan](https://discuss.elastic.co/u/pablosan)\
**Post date:** [April 21, 2017, 9:20am UTC](https://discuss.elastic.co/t/cannot-get-indexes-made-by-logstash/83163/8 "2017-04-21T09:20:45Z")

</div>

Most likely, it's not really running, just checking the config, at least you know your config si good 😄

Please check how are you starting it.

---

<div class="post-metadata">

**Author:** ![sLuvpreet33](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sluvpreet33/32/50626_2.png) [@sLuvpreet33](https://discuss.elastic.co/u/sLuvpreet33)\
**Post date:** [April 21, 2017, 9:29am UTC](https://discuss.elastic.co/t/cannot-get-indexes-made-by-logstash/83163/9 "2017-04-21T09:29:51Z")

</div>

The biggest thing I am not getting is that, port 5044 is opened.

I can check that by `sudo netstat -ntlp`

---

<div class="post-metadata">

**Author:** ![pablosan](https://avatars.discourse-cdn.com/v4/letter/p/e19adc/32.png) [@pablosan](https://discuss.elastic.co/u/pablosan)\
**Post date:** [April 21, 2017, 10:11am UTC](https://discuss.elastic.co/t/cannot-get-indexes-made-by-logstash/83163/10 "2017-04-21T10:11:12Z")

</div>

ps aux | grep logstash?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 19, 2017, 10:19am UTC](https://discuss.elastic.co/t/cannot-get-indexes-made-by-logstash/83163/11 "2017-05-19T10:19:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.