# Create automatic indexes per client per day automatically

**URL:** <https://discuss.elastic.co/t/create-automatic-indexes-per-client-per-day-automatically/289148>\
**Category:** Elasticsearch\
**Created:** [November 14, 2021, 8:25pm UTC](https://discuss.elastic.co/t/create-automatic-indexes-per-client-per-day-automatically/289148 "2021-11-14T20:25:58Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![stasya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stasya/32/80037_2.png) [@stasya](https://discuss.elastic.co/u/stasya)\
**Post date:** [November 14, 2021, 8:25pm UTC](https://discuss.elastic.co/t/create-automatic-indexes-per-client-per-day-automatically/289148/1 "2021-11-14T20:25:58Z")

</div>

Hi All,

I am writing the following documents structure into my Elasticsearch:

```auto
{
"date":"2021-11-03",
"clientId":"123",
...
<some other fields>
}

```

What I would like to happen, is that whenever a document is written to Elasticsearch,  
a new index is created with the name:

myindex-{clientId}-{date}  
where clientId and date are the values from the document.

If such index already exists, the document will be written to it.

This way, I will have a separate index per client per day automatically.

I have looked into index templates and lifecycle management policy, but could not find a way to put both the date and clientId into the index name automatically.

Is there a way to do that?

Thank you!

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 14, 2021, 9:09pm UTC](https://discuss.elastic.co/t/create-automatic-indexes-per-client-per-day-automatically/289148/2 "2021-11-14T21:09:06Z")

</div>

Welcome to our community! 😃

You would be better off using [ILM](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html) to manage that for you, as it will be much more efficient on resources usage.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 14, 2021, 9:12pm UTC](https://discuss.elastic.co/t/create-automatic-indexes-per-client-per-day-automatically/289148/3 "2021-11-14T21:12:12Z")

</div>

The index name need to be set by the process indexing data to Elasticsearch. Be aware that having lots of small indices is very inefficient and does not scale well, so having daily indices per client might only work for small number of clients.

---

<div class="post-metadata">

**Author:** ![stasya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stasya/32/80037_2.png) [@stasya](https://discuss.elastic.co/u/stasya)\
**Post date:** [November 15, 2021, 5:10am UTC](https://discuss.elastic.co/t/create-automatic-indexes-per-client-per-day-automatically/289148/4 "2021-11-15T05:10:23Z")

</div>

Thanks Mark 🙂  
The way I understand it, ILM can add the date to my index names and handle rollover etc... but I didn't find a way to use it to add the clientId into the index name. Is there any way to do that, or do I have to add the clientId in the process indexing the data, like Christian said, and then can use ILM for adding and managing the date part?

---

<div class="post-metadata">

**Author:** ![stasya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stasya/32/80037_2.png) [@stasya](https://discuss.elastic.co/u/stasya)\
**Post date:** [November 15, 2021, 5:14am UTC](https://discuss.elastic.co/t/create-automatic-indexes-per-client-per-day-automatically/289148/5 "2021-11-15T05:14:38Z")

</div>

Thanks for replying, Christian!  
Currently the data is indexed by Kinesis firehose, which can create index per day, but can not add the clientId into the name (not out of the box, anyway). I guess I would need to add a Lambda to that pipeline to achieve my goal.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 13, 2021, 5:14am UTC](https://discuss.elastic.co/t/create-automatic-indexes-per-client-per-day-automatically/289148/6 "2021-12-13T05:14:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.