# Scaling time series indices

**URL:** <https://discuss.elastic.co/t/scaling-time-series-indices/55135>\
**Category:** Elasticsearch\
**Created:** [July 10, 2016, 5:02pm UTC](https://discuss.elastic.co/t/scaling-time-series-indices/55135 "2016-07-10T17:02:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![shushu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shushu/32/10037_2.png) [@shushu](https://discuss.elastic.co/u/shushu)\
**Post date:** [July 10, 2016, 5:02pm UTC](https://discuss.elastic.co/t/scaling-time-series-indices/55135/1 "2016-07-10T17:02:27Z")

</div>

Hi,  
I am running a time series elasticsearch cluster (on top of AWS service).  
Using template, I create a daily index. 5 shards, 2 replicas, on 10 nodes + 4 masters.  
Once the number of clients went high up - all stopped working - CPU maximum went to 100%, while CPU average kept low (~40%).

My guess is that the main searches are done against the latest days, so it focus on the nodes that has latest data - while the rest stays idle.

My question is - what would be the right scale mechanism ?  
I think that I should by default, in the template, give the maximum number of replicas (10), so latest data will have as many replicas as possible.  
Once data become old - in couple of days - reduce the number of replicas to 2.

Is this sounds like a decent methodology ?  
Any other recommendations ?

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [July 10, 2016, 8:04pm UTC](https://discuss.elastic.co/t/scaling-time-series-indices/55135/2 "2016-07-10T20:04:05Z")

</div>

It is likely a lot more efficient to use bigger hardware (ssds, ram, CPU)  
for the new indices and use forced allocation awareness to control indices  
location.

I don't know how you'd do that with any cloud service.

Replicas scale reads well but not writes because replicas just perform the  
same write as the primary.  
shushu [http://discuss.elastic.co/users/shushu](http://discuss.elastic.co/users/shushu) Shushu Inbar  
[http://discuss.elastic.co/users/shushu](http://discuss.elastic.co/users/shushu)  
July 10

Hi,  
I am running a time series elasticsearch cluster (on top of AWS service).  
Using template, I create a daily index. 5 shards, 2 replicas, on 10 nodes +  
4 masters.  
Once the number of clients went high up - all stopped working - CPU maximum  
went to 100%, while CPU average kept low (~40%).

My guess is that the main searches are done against the latest days, so it  
focus on the nodes that has latest data - while the rest stays idle.

My question is - what would be the right scale mechanism ?  
I think that I should by default, in the template, give the maximum number  
of replicas (10), so latest data will have as many replicas as possible.  
Once data become old - in couple of days - reduce the number of replicas to  
2.

Is this sounds like a decent methodology ?  
Any other recommendations ?

Visit Topic  
[http://discuss.elastic.co/t/scaling-time-series-indices/55135/1](http://discuss.elastic.co/t/scaling-time-series-indices/55135/1) or reply  
to this email to respond.

You are receiving these emails cause you have enabled mailing list mode.

To unsubscribe from these emails, click here  
[http://discuss.elastic.co/email/unsubscribe/218e18220e3ce0d999d16a32978f5dcc3428ad3a81bc02edb9c8a875720d4730](http://discuss.elastic.co/email/unsubscribe/218e18220e3ce0d999d16a32978f5dcc3428ad3a81bc02edb9c8a875720d4730)  
.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:36pm UTC](https://discuss.elastic.co/t/scaling-time-series-indices/55135/3 "2017-07-05T22:36:46Z")

</div>


