# What is the recommendation for indexing and sharding in Elasticsearch in complex use case?

**URL:** <https://discuss.elastic.co/t/what-is-the-recommendation-for-indexing-and-sharding-in-elasticsearch-in-complex-use-case/184116>\
**Category:** Elasticsearch\
**Created:** [June 4, 2019, 8:38am UTC](https://discuss.elastic.co/t/what-is-the-recommendation-for-indexing-and-sharding-in-elasticsearch-in-complex-use-case/184116 "2019-06-04T08:38:38Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![vivek14](https://avatars.discourse-cdn.com/v4/letter/v/3ec8ea/32.png) [@vivek14](https://discuss.elastic.co/u/vivek14)\
**Post date:** [June 4, 2019, 8:38am UTC](https://discuss.elastic.co/t/what-is-the-recommendation-for-indexing-and-sharding-in-elasticsearch-in-complex-use-case/184116/1 "2019-06-04T08:38:38Z")

</div>

Hi All Elasticsearch Gurus,

I am a newbie and learning the Elastic Stack and hence need inputs and thoughts from experts..

Assuming an **Elastic Stack** environment wherein the **Logstash** component parses/transforms data from various sources (web apps, mobile apps, switches, routers, OS logs etc) and ships to Elasticsearch cluster to be further viewed in **Kibana** for monitoring and troubleshooting....

I plan to create index ( in the output section of **logstash conf** file) in the following manner for all input sources

1. webapp-index-yyyy.mm.dd
2. mobileapp-index-yyyy-mm.dd
3. syslog-index-yyyy-mm.dd

and similarly for other sources. (i.e. an index per source per day)

I have some questions for the above use

Q1. Is this a good practice? What are the disadvantages or potential future problems if data grows? ( since no. of indices will increase and replica shards will increase too)  
Q2. What is recommendation for such use case?  
Q3. Is creating an index/source better approach? ( e.g. **webapp-index** instead of daily **webapp-index-yyyy.mm.dd** )

Any pointers or suggestions is greatly appreciated.

Thank you in advance,

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 4, 2019, 8:39am UTC](https://discuss.elastic.co/t/what-is-the-recommendation-for-indexing-and-sharding-in-elasticsearch-in-complex-use-case/184116/2 "2019-06-04T08:39:49Z")

</div>

You should look at ILM - [https://www.elastic.co/guide/en/elasticsearch///reference/current/index-lifecycle-management.html](https://www.elastic.co/guide/en/elasticsearch///reference/current/index-lifecycle-management.html)

It's the best way forward!

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [June 4, 2019, 9:01am UTC](https://discuss.elastic.co/t/what-is-the-recommendation-for-indexing-and-sharding-in-elasticsearch-in-complex-use-case/184116/3 "2019-06-04T09:01:05Z")

</div>

I think it depend on capacity of data. for example i have webapp index with 50gb log per day, i will use webapp-yyyy.xx (one index per week).  
and if i have mobileapp with 500mb log per day, i will use mobileapp-yyyy.mm.  
I dont like use yyyy.mm.dd 😃

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 2, 2019, 9:01am UTC](https://discuss.elastic.co/t/what-is-the-recommendation-for-indexing-and-sharding-in-elasticsearch-in-complex-use-case/184116/4 "2019-07-02T09:01:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.