Index load distribution

Hello,

I have been using ES for a year and I am very happy with the performance
and its powerful filtering, but now I have to index 50millions records and
I am a bit worried by the performance in general.

Attending to the doc I undestand that if I setup two new nodes (aws) more
for that new big index, all the caching/load for that index will be held
only in that two new nodes.

There hare three main (<3gb) indexes which are performing well with lots of
queries and this performance should not be degraded by adding the new index.

I would like to confirm if my assumption is correct or if some of the load
for that index searches/cache/etc will be passed to the other nodes which
manages the three main indexes.

Thank you all very much in advance!!

--
Ivan
DevOp - IT Manager at Trovimap.com

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/31d1e52e-194d-40bb-b33f-a86ffdd8b9a8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

ES scales horizontally, so the more nodes you add the better it can perform.
So yes your logic is correct, though you are better off using three nodes
as it minimises the risk of a split cluster.

On 21 November 2014 21:00, Iván González igonzalezvaliente@gmail.com
wrote:

Hello,

I have been using ES for a year and I am very happy with the performance
and its powerful filtering, but now I have to index 50millions records and
I am a bit worried by the performance in general.

Attending to the doc I undestand that if I setup two new nodes (aws) more
for that new big index, all the caching/load for that index will be held
only in that two new nodes.

There hare three main (<3gb) indexes which are performing well with lots
of queries and this performance should not be degraded by adding the new
index.

I would like to confirm if my assumption is correct or if some of the load
for that index searches/cache/etc will be passed to the other nodes which
manages the three main indexes.

Thank you all very much in advance!!

--
Ivan
DevOp - IT Manager at Trovimap.com

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/31d1e52e-194d-40bb-b33f-a86ffdd8b9a8%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/31d1e52e-194d-40bb-b33f-a86ffdd8b9a8%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZkKF0THEc1pKc30MGRZk4_5C1kcgLDmtjADEqZOxXdm1A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

To add to Mark's comment - you'll obviously want to makes sure your cluster
is more or less balanced (in terms of shards, their sizes, etc.). Should
happen automatically, but we've seen a number of situations where things
were not working well because shards were not quite balanced, so you may
want to have a peek at this after you add more nodes.

Otis

Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/

On Friday, November 21, 2014 5:21:27 PM UTC-5, Mark Walkom wrote:

ES scales horizontally, so the more nodes you add the better it can
perform.
So yes your logic is correct, though you are better off using three nodes
as it minimises the risk of a split cluster.

On 21 November 2014 21:00, Iván González <igonzale...@gmail.com
<javascript:>> wrote:

Hello,

I have been using ES for a year and I am very happy with the performance
and its powerful filtering, but now I have to index 50millions records and
I am a bit worried by the performance in general.

Attending to the doc I undestand that if I setup two new nodes (aws) more
for that new big index, all the caching/load for that index will be held
only in that two new nodes.

There hare three main (<3gb) indexes which are performing well with lots
of queries and this performance should not be degraded by adding the new
index.

I would like to confirm if my assumption is correct or if some of the
load for that index searches/cache/etc will be passed to the other nodes
which manages the three main indexes.

Thank you all very much in advance!!

--
Ivan
DevOp - IT Manager at Trovimap.com

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/31d1e52e-194d-40bb-b33f-a86ffdd8b9a8%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/31d1e52e-194d-40bb-b33f-a86ffdd8b9a8%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/23883053-38ac-4b0a-b8d6-54f6c8ac1013%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.