# Logstash does not use the correct index with Elasticsearch output

**URL:** https://discuss.elastic.co/t/logstash-does-not-use-the-correct-index-with-elasticsearch-output/293052
**Category:** Logstash
**Created:** [December 28, 2021, 3:12pm UTC](https://discuss.elastic.co/t/logstash-does-not-use-the-correct-index-with-elasticsearch-output/293052 "2021-12-28T15:12:53Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![dat\_tang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dat_tang/32/99611_2.png) [@dat\_tang](https://discuss.elastic.co/u/dat_tang)
#### Post date: [December 28, 2021, 3:12pm UTC](https://discuss.elastic.co/t/logstash-does-not-use-the-correct-index-with-elasticsearch-output/293052/1 "2021-12-28T15:12:53Z")

</div>

Hi,

I have an ELK stack which I updated 2 weeks ago from 7.2.0 to 7.16.1. The configuration did not change and is as follow:

```auto
# all input will come from filebeat, no local logs
input {
  beats {
    port => 5044
  }
}
filter {
  if [message] =~ /^\{.*\}$/ {
    json {
      source => "message"
    }
  }
  if [ClientHost] {
    geoip {
      source => "ClientHost"
    }
  }
}
output {
  elasticsearch {
      hosts => ["elasticsearch:9200"]
      user => "elastic"
      password => "xxx"
      index => "logstash-%{+yyyy.MM.dd}"
  }    
} 

```

Previously, it worked well. However, now as I check, all the logs are written to the index `logstash`. I could not find where was it set, and I am pretty sure that I didn't change anything.

Does anyone know what is the problem. Thanks a lot in advance.

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [December 28, 2021, 3:18pm UTC](https://discuss.elastic.co/t/logstash-does-not-use-the-correct-index-with-elasticsearch-output/293052/2 "2021-12-28T15:18:09Z")

</div>

Can you explain better what is the issue?

Your Elasticsearch output has the `index` option set to `logstash-%{yyyy.MM.dd}`, so all your logs will be written in daily index starting with `logstash-` and the date will be extracted from the `@timestamp` field, which will be generated by logstash when it received an event.

For examples, for events received today, they will be stored in the index `logstash-2021.12.28`, this is what your configuration does.

---

<div class="post-metadata">

### Author: ![dat\_tang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dat_tang/32/99611_2.png) [@dat\_tang](https://discuss.elastic.co/u/dat_tang)
#### Post date: [December 28, 2021, 3:19pm UTC](https://discuss.elastic.co/t/logstash-does-not-use-the-correct-index-with-elasticsearch-output/293052/3 "2021-12-28T15:19:57Z")

</div>

Hi,

yes, something like `logstash-2021.12.28` is exactly what I expect, and it was working like this when I was using 7.2.0. However, right now, all logs are via Logstash are written to the Elasticsearch index named `logstash`.

---

<div class="post-metadata">

### Author: ![dat\_tang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dat_tang/32/99611_2.png) [@dat\_tang](https://discuss.elastic.co/u/dat_tang)
#### Post date: [December 28, 2021, 3:33pm UTC](https://discuss.elastic.co/t/logstash-does-not-use-the-correct-index-with-elasticsearch-output/293052/4 "2021-12-28T15:33:30Z")

</div>

A quick update from me: I tested by changing the index name to "abc", then the index is created and logs are going to there. I will post my next findings.

---

<div class="post-metadata">

### Author: ![dat\_tang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dat_tang/32/99611_2.png) [@dat\_tang](https://discuss.elastic.co/u/dat_tang)
#### Post date: [December 28, 2021, 4:07pm UTC](https://discuss.elastic.co/t/logstash-does-not-use-the-correct-index-with-elasticsearch-output/293052/6 "2021-12-28T16:07:05Z")

</div>

Ok I found the issue. Took me the whole afternoon:  
Correct:  
`logstash-%{+YYYY.MM.dd}`

What I used:  
`logstash-%{+yyyy.MM.dd}`

Anw, it is funny here: in the 7.2. manual, it tells me to use the "correct" one: [Elasticsearch output plugin | Logstash Reference [7.2] | Elastic](https://www.elastic.co/guide/en/logstash/7.2/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-index)

Meanwhile, the 7.16 manual tells me to use the "wrong" one: [Elasticsearch output plugin | Logstash Reference [7.16] | Elastic](https://www.elastic.co/guide/en/logstash/7.16/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-index)

????

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [December 28, 2021, 5:42pm UTC](https://discuss.elastic.co/t/logstash-does-not-use-the-correct-index-with-elasticsearch-output/293052/7 "2021-12-28T17:42:55Z")

</div>

It does not make much sense and this is probably not the cause of your issue as both `YYYY` and `yyyy` can be used and will give you the same result, which is the four digit year number, like `2021`.

You can check it in the [DateTimeFormat](https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html) class that is used by logstash when parsing the date.

But since it is working now, don' t think if it is worth troubleshooting the issue.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 28, 2021, 6:39pm UTC](https://discuss.elastic.co/t/logstash-does-not-use-the-correct-index-with-elasticsearch-output/293052/8 "2021-12-28T18:39:28Z")

</div>

In 7.16 I would expect the index option to be ignored because it will have [ILM](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-ilm) enabled by default. The default rollover alias is "logstash" which is what you say you were seeing.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 25, 2022, 6:40pm UTC](https://discuss.elastic.co/t/logstash-does-not-use-the-correct-index-with-elasticsearch-output/293052/9 "2022-01-25T18:40:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
