Attribute awareness during recovery?

Hi,

Does anybody know if there is a way, when a node fails and shards need to
be recovered, of configuring ElasticSearch to prefer to recover from a node
sharing the same awareness attributes (e.g. same rack, same zone etc. a bit
like the "automatic preference when searching / getingedit" reference in
the user guide). We're typically seeing a lot of traffic between "zones"
when a failure occurs and I wondered why this was the case / if it was
avoidable. Maybe I'm missing something and recovery always needs to
replicate from the active primary?

Thanks in advance for any guidance or information,

N

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/dfa05e57-15a8-4509-a45f-62db4d94867a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Recovery always needs to come from the primary, otherwise you cannot be
certain your dataset is valid.

On 24 February 2015 at 21:18, Neil Andrassy neil.andrassy@thefilter.com
wrote:

Hi,

Does anybody know if there is a way, when a node fails and shards need to
be recovered, of configuring Elasticsearch to prefer to recover from a node
sharing the same awareness attributes (e.g. same rack, same zone etc. a bit
like the "automatic preference when searching / getingedit" reference in
the user guide). We're typically seeing a lot of traffic between "zones"
when a failure occurs and I wondered why this was the case / if it was
avoidable. Maybe I'm missing something and recovery always needs to
replicate from the active primary?

Thanks in advance for any guidance or information,

N

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dfa05e57-15a8-4509-a45f-62db4d94867a%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dfa05e57-15a8-4509-a45f-62db4d94867a%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_ihD_AkLgoF5FjAJbju9dnCAwAYhoZbns-gf%2BWYXWQMQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

That seems like a shame given the immutability of the underlying blocks.
Sure, the primary needs to identify the specific set of blocks to be
replicated but I don't see why the block data itself couldn't be pulled
from a "closer" replica if it exists there?

On 25 February 2015 at 06:56, Mark Walkom markwalkom@gmail.com wrote:

Recovery always needs to come from the primary, otherwise you cannot be
certain your dataset is valid.

On 24 February 2015 at 21:18, Neil Andrassy neil.andrassy@thefilter.com
wrote:

Hi,

Does anybody know if there is a way, when a node fails and shards need to
be recovered, of configuring Elasticsearch to prefer to recover from a node
sharing the same awareness attributes (e.g. same rack, same zone etc. a bit
like the "automatic preference when searching / getingedit" reference in
the user guide). We're typically seeing a lot of traffic between "zones"
when a failure occurs and I wondered why this was the case / if it was
avoidable. Maybe I'm missing something and recovery always needs to
replicate from the active primary?

Thanks in advance for any guidance or information,

N

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dfa05e57-15a8-4509-a45f-62db4d94867a%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dfa05e57-15a8-4509-a45f-62db4d94867a%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/txa_4eosq0k/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_ihD_AkLgoF5FjAJbju9dnCAwAYhoZbns-gf%2BWYXWQMQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_ihD_AkLgoF5FjAJbju9dnCAwAYhoZbns-gf%2BWYXWQMQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Neil Andrassy | CTO | The Filter
phone | +44 (0)1225 588 004
skype | andrassynp

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CABpTWLObLq3tJuE48p0OfdwoXB6KoAHZvKbi6x%2B98SjuPfrP7A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Define closer though, someone might use zone awareness on a rack or room
level, these are just abstracted concepts/tags after all.

On 25 February 2015 at 23:35, Neil Andrassy (The Filter) <
neil.andrassy@thefilter.com> wrote:

That seems like a shame given the immutability of the underlying blocks.
Sure, the primary needs to identify the specific set of blocks to be
replicated but I don't see why the block data itself couldn't be pulled
from a "closer" replica if it exists there?

On 25 February 2015 at 06:56, Mark Walkom markwalkom@gmail.com wrote:

Recovery always needs to come from the primary, otherwise you cannot be
certain your dataset is valid.

On 24 February 2015 at 21:18, Neil Andrassy neil.andrassy@thefilter.com
wrote:

Hi,

Does anybody know if there is a way, when a node fails and shards need
to be recovered, of configuring Elasticsearch to prefer to recover from a
node sharing the same awareness attributes (e.g. same rack, same zone etc.
a bit like the "automatic preference when searching / getingedit" reference
in the user guide). We're typically seeing a lot of traffic between "zones"
when a failure occurs and I wondered why this was the case / if it was
avoidable. Maybe I'm missing something and recovery always needs to
replicate from the active primary?

Thanks in advance for any guidance or information,

N

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dfa05e57-15a8-4509-a45f-62db4d94867a%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dfa05e57-15a8-4509-a45f-62db4d94867a%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/txa_4eosq0k/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_ihD_AkLgoF5FjAJbju9dnCAwAYhoZbns-gf%2BWYXWQMQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_ihD_AkLgoF5FjAJbju9dnCAwAYhoZbns-gf%2BWYXWQMQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Neil Andrassy | CTO | The Filter
phone | +44 (0)1225 588 004
skype | andrassynp

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CABpTWLObLq3tJuE48p0OfdwoXB6KoAHZvKbi6x%2B98SjuPfrP7A%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CABpTWLObLq3tJuE48p0OfdwoXB6KoAHZvKbi6x%2B98SjuPfrP7A%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_ymUMvG0VXH29_q-_LN0fM0cV1hgxpvVSZEAS10XAg1Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.