# How to collapse individual indices into fewer?

**URL:** <https://discuss.elastic.co/t/how-to-collapse-individual-indices-into-fewer/209619>\
**Category:** Elasticsearch\
**Created:** [November 27, 2019, 7:03am UTC](https://discuss.elastic.co/t/how-to-collapse-individual-indices-into-fewer/209619 "2019-11-27T07:03:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![babarsiddique](https://avatars.discourse-cdn.com/v4/letter/b/53a042/32.png) [@babarsiddique](https://discuss.elastic.co/u/babarsiddique)\
**Post date:** [November 27, 2019, 7:03am UTC](https://discuss.elastic.co/t/how-to-collapse-individual-indices-into-fewer/209619/1 "2019-11-27T07:03:46Z")

</div>

```
Error: Bad Gateway
        at respond (http://xxx.xxx.xxx:5000/bundles/kibana.bundle.js?v=15543:13:2730)
        ........

```

Opening kibana's dashboard stopped the elasticsearch master and elasticsearch health goes from green to RED. It happens whenever the shards are more than approx 8000.

I have set **index\_number\_shards** and **index\_number\_of\_replicas** to **1**. Attached screenshot shows the details of my cluster. 1 elasticsearch master and 2 elasticsearch datanode....elasticsearch master is also act as elasticsearch datanode. Heap size for both nodes are 16g.

![Nodes](https://us1.discourse-cdn.com/elastic/original/3X/5/d/5d6d6a5c8b6a6600d0dc54668cb2d1930234adb8.png)

 ![shards](https://us1.discourse-cdn.com/elastic/original/3X/4/4/44535b54421ecb1ec48f28067f7247141be3a26c.png)  
Any one who can help me to resolve it?  
Thanks

---

<div class="post-metadata">

**Author:** ![mw\_jko](https://avatars.discourse-cdn.com/v4/letter/m/db5fbb/32.png) [@mw\_jko](https://discuss.elastic.co/u/mw_jko)\
**Post date:** [November 27, 2019, 11:50am UTC](https://discuss.elastic.co/t/how-to-collapse-individual-indices-into-fewer/209619/2 "2019-11-27T11:50:58Z")

</div>

Some questions/hints:

It looks like it's still 5 shards per index? (Maybe I read the screenshot wrong) what does `http(s)://yourelasticsearch:9200/specific-index-with-no-wildcards/_count` return? (It should say X count, 1 shard, 1 successful)

Someone told me 1 shard is sufficient for storing ~25gb of data, so with your few megs per index, anything above 1 shard would be wasted. Probably because of this they changed the default from 5 to 1 shard per index with 7.x. The more shards, the more ram hungry the system becomes. I think(!) 800 shards is way too much for this size.

Do you delete old indices?

If possible try not to index into daily indices or aggregate the daily indices in monthly (or even bigger) ones after some time.

You can either do this aggregation by your own (check the `_reindex`-API) or check the Index Lifecycle Policy feature or maybe the Rollup feature. (Because I'm using Elasticsearch since 5.x I went the `_reindex`-way back then, the other features are new to me but look pretty cool and more advanced)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 25, 2019, 11:50am UTC](https://discuss.elastic.co/t/how-to-collapse-individual-indices-into-fewer/209619/3 "2019-12-25T11:50:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.