# One index vs multiple indexes?

**URL:** <https://discuss.elastic.co/t/one-index-vs-multiple-indexes/165984>\
**Category:** Elasticsearch\
**Created:** [January 28, 2019, 10:56am UTC](https://discuss.elastic.co/t/one-index-vs-multiple-indexes/165984 "2019-01-28T10:56:05Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![akhisar](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@akhisar](https://discuss.elastic.co/u/akhisar)\
**Post date:** [January 28, 2019, 10:56am UTC](https://discuss.elastic.co/t/one-index-vs-multiple-indexes/165984/1 "2019-01-28T10:56:05Z")

</div>

Hello All,

Which is a better approach: single index or should it to be split into multiple index. Please can you point out the pros and cons of both approaches.

Rgds,  
-Akhil

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [January 28, 2019, 10:59am UTC](https://discuss.elastic.co/t/one-index-vs-multiple-indexes/165984/2 "2019-01-28T10:59:24Z")

</div>

That generally depends on the use-case, so you will need to provide some additional information if you want anything but very general advice.

---

<div class="post-metadata">

**Author:** ![akhisar](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@akhisar](https://discuss.elastic.co/u/akhisar)\
**Post date:** [January 28, 2019, 12:29pm UTC](https://discuss.elastic.co/t/one-index-vs-multiple-indexes/165984/3 "2019-01-28T12:29:09Z")

</div>

Say, I have environments- dev, staging and production. Currently all the indexes are going to one index pattern logstash-\* on Kibana. It looks that the searches are quite slow. Instead if 3 different index patterns are created like logstashdev-\* ,logstashstagging-\* and logstashprod-\* will it improve the search performance?

Is there any tradeoff if so?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [January 28, 2019, 12:58pm UTC](https://discuss.elastic.co/t/one-index-vs-multiple-indexes/165984/4 "2019-01-28T12:58:52Z")

</div>

Creating separate indices for different types of environments may be a good idea as they could very well have different retention requirements, which often is managed at the index level. Whether it will help with performance will depend on the reason performance is bad. Do you have slow storage? Do you have too much data for the hardware you have? Is the cluster under heavy load? Do you have a lot of small shards and indices?

---

<div class="post-metadata">

**Author:** ![akhisar](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@akhisar](https://discuss.elastic.co/u/akhisar)\
**Post date:** [January 29, 2019, 7:19am UTC](https://discuss.elastic.co/t/one-index-vs-multiple-indexes/165984/5 "2019-01-29T07:19:43Z")

</div>

There are around 30M documents per day. We have a 7 node cluster on azure- 1 Client, 3 data and 3 master each of 16GB RAM and 2CPUs per node. 3 Data nodes are connected with Azure premium SSD disks. Current retention period is 2 weeks.

Currently all daily micro-service log are going to the same index logstash-. Which means only on index per day. Default settings is 5 Shards and 1 replica.

If the hardware is not sufficient could you please help on how to do a sizing in here.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 29, 2019, 7:34am UTC](https://discuss.elastic.co/t/one-index-vs-multiple-indexes/165984/6 "2019-01-29T07:34:35Z")

</div>

You probably have too many shards per node.

May I suggest you look at the following resources about sizing:

[https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing](https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing)

> **[How many shards should I have in my Elasticsearch cluster?
	  	 | Elastic](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster)**
>
> Elasticsearch is a very versatile platform, that supports a variety of use cases, and provides great flexibility around data organisation and replication strategies. This flexibility can however somet...

> **[NetSecureDay: Managing your Black Friday Logs](https://speakerdeck.com/elastic/netsecureday-managing-your-black-friday-logs)**
>
> Surveiller une application complexe n’est pas une tâche aisée, mais avec les bons outils, ce n’est pas si sorcier. Néanmoins, des périodes fortes telles que les opérations de type « Black Friday » (Vendredi noir) ou période de Noël peuvent pousser...

And [https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right](https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right)

---

<div class="post-metadata">

**Author:** ![akhisar](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@akhisar](https://discuss.elastic.co/u/akhisar)\
**Post date:** [January 29, 2019, 8:32am UTC](https://discuss.elastic.co/t/one-index-vs-multiple-indexes/165984/7 "2019-01-29T08:32:17Z")

</div>

Thanks 🙂

I will check on these and make the necessary changes.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 26, 2019, 8:32am UTC](https://discuss.elastic.co/t/one-index-vs-multiple-indexes/165984/8 "2019-02-26T08:32:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
