# Daily Indexing (at Midnight)

**URL:** <https://discuss.elastic.co/t/daily-indexing-at-midnight/272867>\
**Category:** Elasticsearch\
**Created:** [May 13, 2021, 1:04am UTC](https://discuss.elastic.co/t/daily-indexing-at-midnight/272867 "2021-05-13T01:04:49Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![inf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inf/32/85793_2.png) [@inf](https://discuss.elastic.co/u/inf)\
**Post date:** [May 13, 2021, 1:04am UTC](https://discuss.elastic.co/t/daily-indexing-at-midnight/272867/1 "2021-05-13T01:04:50Z")

</div>

Hi there,

As I understand from this [GitHub post](https://github.com/elastic/elasticsearch/issues/47838) and this [Elastic Discuss post](https://discuss.elastic.co/t/ilm-rotate-on-midnight/196279) about having a new daily index (at midnight for my case), the solution to it is not Index Lifecycle Management. Instead, it should be using date math index.

I am trying this out, and using Winlogbeat as a Beats example.

I would like to ask if it is just the simple step of adding these 3 lines in my `winlogbeat.yml` configuration file?

Line #1:  
`setup.template.name: "winlogbeat-test"`

Line #2:  
`setup.template.pattern: "winlogbeat-test-*"`

Line #3:  
`output.elasticsearch.index: "winlogbeat-%{[agent.version]}-%{+yyyy.MM.dd}"`

The documentation reference page which I looked at for this is found [here](https://www.elastic.co/guide/en/beats/winlogbeat/current/change-index-name.html).

(I have no need for a custom name, and I would like to have the agent's version as part of the index name.)

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 13, 2021, 1:44am UTC](https://discuss.elastic.co/t/daily-indexing-at-midnight/272867/2 "2021-05-13T01:44:44Z")

</div>

Is there a reason for not using ILM?

---

<div class="post-metadata">

**Author:** ![inf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inf/32/85793_2.png) [@inf](https://discuss.elastic.co/u/inf)\
**Post date:** [May 13, 2021, 2:10am UTC](https://discuss.elastic.co/t/daily-indexing-at-midnight/272867/3 "2021-05-13T02:10:41Z")

</div>

It was mentioned in the Elastic Discuss post that:

> [@](#):
>
> The whole point of using rollover is to allow indices to roll over based on size and/or age, which goes against specifying a time. If you want your indices to roll over at a specific time then set your index names accordingly and do not use rollover.

I agree with it (its reasoning appeared sound to me). Moreover, what ILM offers is an age-based rollover, which isn't what I am looking for.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 13, 2021, 2:12am UTC](https://discuss.elastic.co/t/daily-indexing-at-midnight/272867/4 "2021-05-13T02:12:22Z")

</div>

Yep, but I am just looking for why you want date based indices?

---

<div class="post-metadata">

**Author:** ![inf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inf/32/85793_2.png) [@inf](https://discuss.elastic.co/u/inf)\
**Post date:** [May 13, 2021, 2:14am UTC](https://discuss.elastic.co/t/daily-indexing-at-midnight/272867/5 "2021-05-13T02:14:20Z")

</div>

Right. Thank you for the clarification - this might sound silly, but I would like to find out how many documents are held in an index each day and the size that a daily batch worth of document's holds.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 13, 2021, 2:20am UTC](https://discuss.elastic.co/t/daily-indexing-at-midnight/272867/6 "2021-05-13T02:20:15Z")

</div>

That's not silly, makes total sense 🙂 It's a limitation of ILM.  
FWIW I raised [Show per day size stats for ILM indices · Issue #100009 · elastic/kibana · GitHub](https://github.com/elastic/kibana/issues/100009) to see if we can do something here.

Basically if you want this via Beats, just disable the [ILM portion](https://www.elastic.co/guide/en/beats/metricbeat/current/ilm.html) of the config.

---

<div class="post-metadata">

**Author:** ![inf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inf/32/85793_2.png) [@inf](https://discuss.elastic.co/u/inf)\
**Post date:** [May 13, 2021, 2:25am UTC](https://discuss.elastic.co/t/daily-indexing-at-midnight/272867/7 "2021-05-13T02:25:17Z")

</div>

Just found out that interestingly someone else raised the same underlying question just very recently!

Thank you so much for this. 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 10, 2021, 2:25am UTC](https://discuss.elastic.co/t/daily-indexing-at-midnight/272867/8 "2021-06-10T02:25:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
