# Elastic not creating index daily based on date

**URL:** https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753
**Category:** Logstash
**Created:** [June 4, 2020, 12:39pm UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753 "2020-06-04T12:39:25Z")
**Posts on this page:** 14
**Page:** 1

<div class="post-metadata">

### Author: ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)
#### Post date: [June 4, 2020, 12:39pm UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/1 "2020-06-04T12:39:25Z")

</div>

I use Logstash for ingestion in elastic but my index not creating index daily based on date

my config logstsh

```
input {
beats {
port => "5044"
}
}  

filter {

csv {

separator => ","
skip_header => "true"
columns => ["chaine", "job", "date_plan", "statut", "date_debut", "date_fin", "serveur", "numero_passage", "application", "sous_application"]

}

date {
match => ["date_plan" , "YYYY-MM-dd"]
timezone => "Europe/Paris"
}

date {
match => ["date_debut" , "YYYY-MM-dd HH:mm:ss"]
timezone => "Europe/Paris"
}

date {
match => ["date_fin" , "YYYY-MM-dd HH:mm:ss"]
timezone => "Europe/Paris"
}

mutate {
convert => { "numero_passage" => "integer" }
}

}

output {
elasticsearch {
hosts => ["http://elasticsearch:9200"]
index => "test-%{+YYYY.MM.dd}"  
stdout {
codec => rubydebug
}

}
```

---

<div class="post-metadata">

### Author: ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)
#### Post date: [June 4, 2020, 2:21pm UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/2 "2020-06-04T14:21:39Z")

</div>

Hi @Youssef_SBAI,

do you see any errors on `stdout`?

The only thing I can think of is that you are missing the `@timestamp` field...

From the [docs](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-index)

```
LS uses Joda to format the index pattern from event timestamp.
```

---

<div class="post-metadata">

### Author: ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)
#### Post date: [June 4, 2020, 2:24pm UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/3 "2020-06-04T14:24:55Z")

</div>

No i have any errror in stdout

---

<div class="post-metadata">

### Author: ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)
#### Post date: [June 4, 2020, 2:35pm UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/4 "2020-06-04T14:35:54Z")

</div>

Do you have a field named `@timestamp` in the `stdout` output?

What are your indices named? If there is no error there should have been some indices created...

---

<div class="post-metadata">

### Author: ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)
#### Post date: [June 4, 2020, 2:37pm UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/5 "2020-06-04T14:37:31Z")

</div>

yes i have a field named `@timestamp` in the `stdout` output

---

<div class="post-metadata">

### Author: ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)
#### Post date: [June 4, 2020, 2:39pm UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/6 "2020-06-04T14:39:41Z")

</div>

I have auther index created test- 2020.05.29 test- 2020.05.28 test- 2020.05.27 ..........

---

<div class="post-metadata">

### Author: ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)
#### Post date: [June 5, 2020, 7:18am UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/7 "2020-06-05T07:18:38Z")

</div>

These index names

> [@Youssef\_SBAI](#):
>
> test- 2020.05.21 test- 2020.05.20 test- 2020.05.19

Match exactly your Logstash output config with

> [@Youssef\_SBAI](#):
>
> `index => "test-%{+YYYY.MM.dd}"`

Those are the daily indices. The date for the index name is taken from the `@timestamp` field, not current time on the Logstash machine.

Could you share the content of one of those `@timestamp` fields

---

<div class="post-metadata">

### Author: ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)
#### Post date: [June 5, 2020, 7:31am UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/8 "2020-06-05T07:31:11Z")

</div>

The content of `@timestamp` fields is

![image](https://us1.discourse-cdn.com/elastic/original/3X/2/a/2a3126de963b984436f86ff990f2db7e8a43d889.png)

---

<div class="post-metadata">

### Author: ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)
#### Post date: [June 5, 2020, 7:34am UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/9 "2020-06-05T07:34:49Z")

</div>

Thanks 🙂

Those logs would end up in an index with the name `test-2020.05.29`, no matter _when_ they pass through Logstash.

---

<div class="post-metadata">

### Author: ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)
#### Post date: [June 5, 2020, 7:40am UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/10 "2020-06-05T07:40:42Z")

</div>

I have an other content with the index

test-2020.05.29

![image](https://us1.discourse-cdn.com/elastic/original/3X/4/3/43636a1aa9c62be09034ea1d7229854d3339ed62.png)

---

<div class="post-metadata">

### Author: ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)
#### Post date: [June 5, 2020, 7:41am UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/11 "2020-06-05T07:41:21Z")

</div>

it is a problem of `@timestamp` fields?

---

<div class="post-metadata">

### Author: ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)
#### Post date: [June 5, 2020, 7:59am UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/12 "2020-06-05T07:59:09Z")

</div>

If you want to call it a _problem_, I guess yes.

As I posted before

> [@A\_B](#):
>
> `Logstash uses Joda to format the index pattern from event timestamp.`

`event timestamp` = `@timestamp` field.

If you would rather have `now` (current time) instead of the event timestamp, you can replace `@timestamp` with a filter. Simply removing `@timestamp` might have the same result, not 100% sure about that.

---

<div class="post-metadata">

### Author: ![Youssef\_SBAI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youssef_sbai/32/64242_2.png) [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)
#### Post date: [June 8, 2020, 6:17am UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/13 "2020-06-08T06:17:02Z")

</div>

I use Joda to format the index: index =\> "\<logstash-{now/d}\>" Thank You

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2020, 6:17am UTC](https://discuss.elastic.co/t/elastic-not-creating-index-daily-based-on-date/235753/14 "2020-07-06T06:17:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
