# Unable to create elasticsearch index from logstash.conf file

**URL:** https://discuss.elastic.co/t/unable-to-create-elasticsearch-index-from-logstash-conf-file/225693
**Category:** Elasticsearch
**Created:** [March 30, 2020, 2:18pm UTC](https://discuss.elastic.co/t/unable-to-create-elasticsearch-index-from-logstash-conf-file/225693 "2020-03-30T14:18:25Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![sachinarora](https://avatars.discourse-cdn.com/v4/letter/s/a88e57/32.png) [@sachinarora](https://discuss.elastic.co/u/sachinarora)
#### Post date: [March 30, 2020, 2:18pm UTC](https://discuss.elastic.co/t/unable-to-create-elasticsearch-index-from-logstash-conf-file/225693/1 "2020-03-30T14:18:25Z")

</div>

I am trying to create index of elasticsearch using logstash.conf file but I am not able to create index. There is not any error in logs of elasticsearch and logstash. After starting the logstash whenever I do url/\_cat/indices it is not showing index in the list but whenever I am commenting index tag from conf file it is creating the default index with name logstash-%(date)  
Pls help me I am new to this.  
Following is the logstash.conf file

```auto
input {
  file {
    path => ["/var/lib/jenkins/jobs/settlement_poc/builds/124/log"]
    type => "syslog"
  }
}
output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "syslog-%{+YYYY.MM.dd}"
    document_type => "system_logs"
  }
}

```

---

<div class="post-metadata">

### Author: ![Mike.Barretta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike.barretta/32/16688_2.png) [@Mike.Barretta](https://discuss.elastic.co/u/Mike.Barretta)
#### Post date: [April 1, 2020, 2:00pm UTC](https://discuss.elastic.co/t/unable-to-create-elasticsearch-index-from-logstash-conf-file/225693/2 "2020-04-01T14:00:40Z")

</div>

@sachinarora I'm not positive, but you should do two things:

1. Try running your pipeline with `output { stdout {} }` to make sure you're getting records emitted at all
2. Try running without setting `document_type`: that is a deprecated field and should be using the default value of `_doc`. See [https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-document\_type](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-document_type)

---

<div class="post-metadata">

### Author: ![sachinarora](https://avatars.discourse-cdn.com/v4/letter/s/a88e57/32.png) [@sachinarora](https://discuss.elastic.co/u/sachinarora)
#### Post date: [April 2, 2020, 12:49pm UTC](https://discuss.elastic.co/t/unable-to-create-elasticsearch-index-from-logstash-conf-file/225693/3 "2020-04-02T12:49:59Z")

</div>

yes i tried running with stdout it is printing logs and I also removed document type still same issue

---

<div class="post-metadata">

### Author: ![sachinarora](https://avatars.discourse-cdn.com/v4/letter/s/a88e57/32.png) [@sachinarora](https://discuss.elastic.co/u/sachinarora)
#### Post date: [April 2, 2020, 1:45pm UTC](https://discuss.elastic.co/t/unable-to-create-elasticsearch-index-from-logstash-conf-file/225693/4 "2020-04-02T13:45:32Z")

</div>

Hey what I found it is creating index of one log file file but not for another  
second log file does n't have any timestamp does it matter while creating index

---

<div class="post-metadata">

### Author: ![Mike.Barretta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike.barretta/32/16688_2.png) [@Mike.Barretta](https://discuss.elastic.co/u/Mike.Barretta)
#### Post date: [April 2, 2020, 2:18pm UTC](https://discuss.elastic.co/t/unable-to-create-elasticsearch-index-from-logstash-conf-file/225693/5 "2020-04-02T14:18:48Z")

</div>

and index doesn't need a timestamp or date field, no. But, if the second log file isn't in syslog format, you input codec might be causing the problem. Do you see any errors in Logstash logs?

---

<div class="post-metadata">

### Author: ![sachinarora](https://avatars.discourse-cdn.com/v4/letter/s/a88e57/32.png) [@sachinarora](https://discuss.elastic.co/u/sachinarora)
#### Post date: [April 3, 2020, 7:06am UTC](https://discuss.elastic.co/t/unable-to-create-elasticsearch-index-from-logstash-conf-file/225693/6 "2020-04-03T07:06:42Z")

</div>

no there is not any error in logstash and both log files are completely different , I did not created any format for any log file..I just used another log file for testing purpose. But it creating index for one file but not creating for the file I need.

---

<div class="post-metadata">

### Author: ![Mike.Barretta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike.barretta/32/16688_2.png) [@Mike.Barretta](https://discuss.elastic.co/u/Mike.Barretta)
#### Post date: [April 3, 2020, 1:37pm UTC](https://discuss.elastic.co/t/unable-to-create-elasticsearch-index-from-logstash-conf-file/225693/7 "2020-04-03T13:37:30Z")

</div>

I highly recommend that you give a close reading to the [file input documentation](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.htm). For example, are you in `read` mode or `tail` mode? Are you aware of the implications of `sincedb`? If using the default `tail` mode, is the second log file receiving new events or are you expecting Logstash to process it from the beginning, which means you need to set `start_position`?

For a sanity check, run a pipeline like:

```auto
input {
    file {
        path => "/var/lib/jenkins/jobs/settlement_poc/builds/124/log"
        start_position => "beginning"
        sincedb_path => "/dev/null"
    }
}
output {
    stdout {}
}

```

...and do it at TRACE level logging: `logtash --log.level TRACE -f <your pipeline>`

You should see TRACE message for every file it finds (look for log lines with "filewatch.discoverer").

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 1, 2020, 1:37pm UTC](https://discuss.elastic.co/t/unable-to-create-elasticsearch-index-from-logstash-conf-file/225693/8 "2020-05-01T13:37:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
