# Scaling elasticsearch queries **URL:** \ **Category:** Elasticsearch\ **Created:** [October 24, 2013, 5:51pm UTC](https://discuss.elastic.co/t/scaling-elasticsearch-queries/14097 "2013-10-24T17:51:38Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Milad\_Fatenejad](https://avatars.discourse-cdn.com/v4/letter/m/e95f7d/32.png) [@Milad\_Fatenejad](https://discuss.elastic.co/u/Milad_Fatenejad)\ **Post date:** [October 24, 2013, 5:51pm UTC](https://discuss.elastic.co/t/scaling-elasticsearch-queries/14097/1 "2013-10-24T17:51:38Z")
Hello: I had a question related to handling increased query demand. Let's say I have an index which stores data for a single month which is queried at some rate. Then a few months later, due to some kind of external event, there is a substantially increased interest in searching this data. How do I scale up the index to handle the larger number of queries? At this point I am not indexing any new documents, just searching. I know that I can add new nodes and shards will be moved there, but this stops working once you have 1 shard per node. At this point, what do I do? Do I increase the number of replicas? If so, is it easy to later (after the demand drops) decrease the number of replicas? Thank You Milad -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 24, 2013, 8:44pm UTC](https://discuss.elastic.co/t/scaling-elasticsearch-queries/14097/2 "2013-10-24T20:44:43Z")
Yes. Exactly. You will perhaps have to provide more machines as well to hold all your replicas. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 24 oct. 2013 à 19:51, Milad Fatenejad [icksa1@gmail.com](mailto:icksa1@gmail.com) a écrit : > Hello: > > I had a question related to handling increased query demand. Let's say I have an index which stores data for a single month which is queried at some rate. Then a few months later, due to some kind of external event, there is a substantially increased interest in searching this data. How do I scale up the index to handle the larger number of queries? At this point I am not indexing any new documents, just searching. I know that I can add new nodes and shards will be moved there, but this stops working once you have 1 shard per node. > > At this point, what do I do? Do I increase the number of replicas? If so, is it easy to later (after the demand drops) decrease the number of replicas? > > ## Thank You Milad > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Milad\_Fatenejad](https://avatars.discourse-cdn.com/v4/letter/m/e95f7d/32.png) [@Milad\_Fatenejad](https://discuss.elastic.co/u/Milad_Fatenejad)\ **Post date:** [October 24, 2013, 9:07pm UTC](https://discuss.elastic.co/t/scaling-elasticsearch-queries/14097/3 "2013-10-24T21:07:59Z")
Thank you Dave, one quick follow up question. We have two data centers in our cloud and I would like to have the primary shard and replica split across these data centers. I read the following page: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. and I believe the location awareness settings will accomplish this, for example I can set: cluster.routing.allocation.awareness.attributes: datacenter If I add a replica (so my replica count is 2), what will happen with the location awareness? The documentation seems to only say "that a shard and its replica won't share the same" datacenter, so what happens if there are only 2 data centers, but 3 copies of the data (i.e. replica count = 2)? Will all replicas move to one datacenter? Or will elasticsearch try to spread things around as much as possible in some way? Thanks again! Milad On Thu, Oct 24, 2013 at 3:44 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Yes. Exactly. You will perhaps have to provide more machines as well to > hold all your replicas. > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 24 oct. 2013 à 19:51, Milad Fatenejad [icksa1@gmail.com](mailto:icksa1@gmail.com) a écrit : > > Hello: > > I had a question related to handling increased query demand. Let's say I > have an index which stores data for a single month which is queried at some > rate. Then a few months later, due to some kind of external event, there is > a substantially increased interest in searching this data. How do I scale > up the index to handle the larger number of queries? At this point I am not > indexing any new documents, just searching. I know that I can add new nodes > and shards will be moved there, but this stops working once you have 1 > shard per node. > > At this point, what do I do? Do I increase the number of replicas? If so, > is it easy to later (after the demand drops) decrease the number of > replicas? > > Thank You > Milad > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/2a1WGHDKHfY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/2a1WGHDKHfY/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 25, 2013, 7:59am UTC](https://discuss.elastic.co/t/scaling-elasticsearch-queries/14097/4 "2013-10-25T07:59:30Z")
Ha! Good question. I'm not sure but I guess it won't be allocated anywhere. That said, unless you have a very low latency network between the two datacenters, I would not recommend to use that architecture. I would create two clusters and push documents from client application in both clusters. If network connectivity is good enough and guaranteed, then you can probably go for it but keep in mind that you will probably to think carefully of split brain issues when datacenters are disconnected. -- David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) @dadoonet | @elasticsearchfr | @scrutmydocs Le 24 oct. 2013 à 23:07, Milad Fatenejad [icksa1@gmail.com](mailto:icksa1@gmail.com) a écrit : > Thank you Dave, one quick follow up question. We have two data centers in our cloud and I would like to have the primary shard and replica split across these data centers. I read the following page: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html#allocation-awareness) > > and I believe the location awareness settings will accomplish this, for example I can set: > cluster.routing.allocation.awareness.attributes: datacenter > If I add a replica (so my replica count is 2), what will happen with the location awareness? The documentation seems to only say "that a shard and its replica won't share the same" datacenter, so what happens if there are only 2 data centers, but 3 copies of the data (i.e. replica count = 2)? Will all replicas move to one datacenter? Or will elasticsearch try to spread things around as much as possible in some way? > > Thanks again! > Milad > > On Thu, Oct 24, 2013 at 3:44 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Yes. Exactly. You will perhaps have to provide more machines as well to hold all your replicas. > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 24 oct. 2013 à 19:51, Milad Fatenejad [icksa1@gmail.com](mailto:icksa1@gmail.com) a écrit : > > > Hello: > > > > I had a question related to handling increased query demand. Let's say I have an index which stores data for a single month which is queried at some rate. Then a few months later, due to some kind of external event, there is a substantially increased interest in searching this data. How do I scale up the index to handle the larger number of queries? At this point I am not indexing any new documents, just searching. I know that I can add new nodes and shards will be moved there, but this stops working once you have 1 shard per node. > > > > At this point, what do I do? Do I increase the number of replicas? If so, is it easy to later (after the demand drops) decrease the number of replicas? > > > > Thank You > > Milad > > > > -- > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/2a1WGHDKHfY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/2a1WGHDKHfY/unsubscribe). > To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Milad\_Fatenejad](https://avatars.discourse-cdn.com/v4/letter/m/e95f7d/32.png) [@Milad\_Fatenejad](https://discuss.elastic.co/u/Milad_Fatenejad)\ **Post date:** [October 25, 2013, 12:31pm UTC](https://discuss.elastic.co/t/scaling-elasticsearch-queries/14097/5 "2013-10-25T12:31:08Z")
Thank you David, this has been very helpful. Milad On Fri, Oct 25, 2013 at 2:59 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Ha! Good question. I'm not sure but I guess it won't be allocated anywhere. > > That said, unless you have a very low latency network between the two > datacenters, I would not recommend to use that architecture. > I would create two clusters and push documents from client application in > both clusters. > > If network connectivity is good enough and guaranteed, then you can > probably go for it but keep in mind that you will probably to think > carefully of split brain issues when datacenters are disconnected. > > -- > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_ > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) > | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > Le 24 oct. 2013 à 23:07, Milad Fatenejad [icksa1@gmail.com](mailto:icksa1@gmail.com) a écrit : > > Thank you Dave, one quick follow up question. We have two data centers in > our cloud and I would like to have the primary shard and replica split > across these data centers. I read the following page: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html#allocation-awareness) > > and I believe the location awareness settings will accomplish this, for > example I can set: > > cluster.routing.allocation.awareness.attributes: datacenter > > If I add a replica (so my replica count is 2), what will happen with the > location awareness? The documentation seems to only say "that a shard and > its replica won't share the same" datacenter, so what happens if there are > only 2 data centers, but 3 copies of the data (i.e. replica count = 2)? > Will all replicas move to one datacenter? Or will elasticsearch try to > spread things around as much as possible in some way? > > Thanks again! > Milad > > On Thu, Oct 24, 2013 at 3:44 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > > > Yes. Exactly. You will perhaps have to provide more machines as well to > > hold all your replicas. > > > > -- > > David 😉 > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > Le 24 oct. 2013 à 19:51, Milad Fatenejad [icksa1@gmail.com](mailto:icksa1@gmail.com) a écrit : > > > > Hello: > > > > I had a question related to handling increased query demand. Let's say I > > have an index which stores data for a single month which is queried at some > > rate. Then a few months later, due to some kind of external event, there is > > a substantially increased interest in searching this data. How do I scale > > up the index to handle the larger number of queries? At this point I am not > > indexing any new documents, just searching. I know that I can add new nodes > > and shards will be moved there, but this stops working once you have 1 > > shard per node. > > > > At this point, what do I do? Do I increase the number of replicas? If so, > > is it easy to later (after the demand drops) decrease the number of > > replicas? > > > > Thank You > > Milad > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/2a1WGHDKHfY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/2a1WGHDKHfY/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/2a1WGHDKHfY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/2a1WGHDKHfY/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:10am UTC](https://discuss.elastic.co/t/scaling-elasticsearch-queries/14097/6 "2017-07-06T02:10:36Z")