# Need Help: Date-wise Index Creation with Application Using Built-in Elasticsearch and Static Index Name **URL:** \ **Category:** Elasticsearch\ **Tags:** ilm-index-lifecycle-management, reindex\ **Created:** [April 23, 2026, 12:12pm UTC](https://discuss.elastic.co/t/need-help-date-wise-index-creation-with-application-using-built-in-elasticsearch-and-static-index-name/386007 "2026-04-23T12:12:30Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Shubham\_Khodpe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubham_khodpe/32/147372_2.png) [@Shubham\_Khodpe](https://discuss.elastic.co/u/Shubham_Khodpe)\ **Post date:** [April 23, 2026, 12:12pm UTC](https://discuss.elastic.co/t/need-help-date-wise-index-creation-with-application-using-built-in-elasticsearch-and-static-index-name/386007/1 "2026-04-23T12:12:30Z")
**## Environment Details ###** - Elasticsearch (self-managed cluster on AWS) - 4 Data Nodes (1 TB each) - 2 Master Nodes - Daily data ingestion around 300 GB (without replica) and approximately 600 GB with 1 replica - Data source is an application that has built-in Elasticsearch integration * * * ## Current Architecture Application (with built-in Elasticsearch integration) ↓ Elasticsearch Cluster ↓ Index: greenzone-aws\_transactionalevents - The application pushes data directly into Elasticsearch - Elasticsearch connection is configured from the application UI - Index name is defined in the application configuration - The index name remains static * * * ## Problem Statement The issue I am facing is as follows: 1. The application allows configuring only a static index name such as: greenzone-aws\_transactionalevents 2. It does not support dynamic index naming like date-based patterns 3. It does not automatically create or rotate indices based on date * * * ## Impact of Current Setup Due to this limitation, the following challenges are observed: 1. Continuous index growth - Around 300 GB data is ingested daily - The same index keeps growing without any rotation 1. No date-wise data separation - All data is stored in a single index - It is difficult to isolate data for a specific day 1. Snapshot limitations - Snapshots work at index level - It is not possible to take snapshots for specific date ranges within the same index 1. Restore limitations - It is not possible to restore data for a specific date - Full index restore is required even for partial data 1. Operational challenges - Storage management becomes difficult - Large index size can impact performance over time * * * ## Requirement I want to achieve the following: 1. Date-wise index creation, for example: greenzone-aws\_transactionalevents-2026.04.22 greenzone-aws\_transactionalevents-2026.04.23 2. Better data organization - Logical separation of data per day 1. Granular backup and restore - Ability to take snapshots and restore specific date-based data * * * ## Questions 1. Does an application with built-in Elasticsearch integration typically support: - Dynamic index naming - Date-based index creation or rotation 1. Is there any configuration in Elasticsearch that can: - Automatically split or manage data by date when a static index name is used 1. Can index templates, ingest pipelines, or any Elasticsearch feature: - Override or influence index naming when data is pushed from an external application 1. What are the best practices for handling: - High-volume ingestion (around 300 GB per day) - When the source system enforces a static index name * * * ## Summary - The application pushes data to Elasticsearch using a static index name - Dynamic or date-based index naming is not supported at the application level - Need guidance on achieving day-wise index structure for better data management, backup, and restoration * * * Looking for suggestions and best practices to handle this scenario effectively. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [April 23, 2026, 1:24pm UTC](https://discuss.elastic.co/t/need-help-date-wise-index-creation-with-application-using-built-in-elasticsearch-and-static-index-name/386007/2 "2026-04-23T13:24:07Z")
> [@Shubham\_Khodpe](#): > > 1. Does an application with built-in Elasticsearch integration typically support: > > - Dynamic index naming > - Date-based index creation or rotation The index naming is up to the application, it needs to be implemented in the applicaiton code, the roration should be controlled by an Index Lifecycle Policy or if the application changes the name each day. I would also say that rotation based on data is not a good approach, you should aim to rotate based on size of the index. With an index lifecycle policy you can configure it to rotate based on size and age, but not on exact dates. > [@Shubham\_Khodpe](#): > > 1. Is there any configuration in Elasticsearch that can: > > - Automatically split or manage data by date when a static index name is used An Index lifecycle policy can do that as it will have a write alias, which is static, and it uses backing index to write the data, but as mentioned, those indices will rotate based on size or age. > [@Shubham\_Khodpe](#): > > 1. Can index templates, ingest pipelines, or any Elasticsearch feature: > > - Override or influence index naming when data is pushed from an external application Yes, you can configure an `index.final_pipeline` in your index template so it would always call an ingest pipeline in elasticsearch before indexing, in this indexing pipeline you could use some processors to change the index name, but this would depend on your data. > [@Shubham\_Khodpe](#): > > 1. What are the best practices for handling: > > - High-volume ingestion (around 300 GB per day) > - When the source system enforces a static index name The common approach is to use a data stream and rotate by size, not by day, it is easier to manage. But you need to provide more context about this application, does it only write to elasticsearch or does it read from elasticsearch as well? Does it update the data once it is written? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [April 24, 2026, 2:29am UTC](https://discuss.elastic.co/t/need-help-date-wise-index-creation-with-application-using-built-in-elasticsearch-and-static-index-name/386007/3 "2026-04-24T02:29:36Z")
What version Elastic are you on? ---
**Author:** ![Shubham\_Khodpe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubham_khodpe/32/147372_2.png) [@Shubham\_Khodpe](https://discuss.elastic.co/u/Shubham_Khodpe)\ **Post date:** [April 24, 2026, 2:53am UTC](https://discuss.elastic.co/t/need-help-date-wise-index-creation-with-application-using-built-in-elasticsearch-and-static-index-name/386007/4 "2026-04-24T02:53:37Z")
All components on 9.1.3 Also please give documentation if any for such cases. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [April 24, 2026, 3:20am UTC](https://discuss.elastic.co/t/need-help-date-wise-index-creation-with-application-using-built-in-elasticsearch-and-static-index-name/386007/5 "2026-04-24T03:20:05Z")
If you have a static index name in the application I would create a data stream template that matches that that index name Create a custom ILM policy applied to the data stream template Then your application will be writing to a data stream and then all the naming will be taken care of for you And it will use the ilm policy that applied. All the actual backing indices will be managed by the data stream. > **[Data streams | Elastic Docs](https://www.elastic.co/docs/manage-data/data-store/data-streams)** > > A data stream acts as a layer of abstraction over a set of indices that are optimized for storing append-only time series data. It stores data across... ---
**Author:** ![Shubham\_Khodpe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubham_khodpe/32/147372_2.png) [@Shubham\_Khodpe](https://discuss.elastic.co/u/Shubham_Khodpe)\ **Post date:** [May 20, 2026, 7:36am UTC](https://discuss.elastic.co/t/need-help-date-wise-index-creation-with-application-using-built-in-elasticsearch-and-static-index-name/386007/6 "2026-05-20T07:36:12Z")
I have already tried with Data stream approach still it is not working. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [May 20, 2026, 1:24pm UTC](https://discuss.elastic.co/t/need-help-date-wise-index-creation-with-application-using-built-in-elasticsearch-and-static-index-name/386007/7 "2026-05-20T13:24:00Z")
> [@Shubham\_Khodpe](#): > > I have already tried with Data stream approach still it is not working. Some details would help here... What's not working? What is the error? Using a static name for data stream. Is it common use case so it's not clear what your issue is?