Failover with custom search preference


(Patrick Peschlow) #1

The search preference documentation on
http://www.elasticsearch.org/guide/reference/api/search/preference/ states
that a custom (string) preference is supported. However, it does not
explicitly state how possible failovers are handled in that case. Suppose
that a custom preference value "user-1234" is currently (internally)
associated with shard 1. What happens if shard 1 then goes down? WIll
search queries with preference value "user-1234" also fail from that point
onwards, or will it fail over, i.e., those custom preferences affected will
get re-mapped internally to other shards?

Thanks for any insights,
Patrick

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(yarin) #2

I'm also really interested in the answer for Patrick's question.

Bumping this in hope to get some answer.

Thanks in advance,
Yarin.

On Wednesday, August 28, 2013 2:59:14 PM UTC+3, Patrick Peschlow wrote:

The search preference documentation on
http://www.elasticsearch.org/guide/reference/api/search/preference/states that a custom (string) preference is supported. However, it does not
explicitly state how possible failovers are handled in that case. Suppose
that a custom preference value "user-1234" is currently (internally)
associated with shard 1. What happens if shard 1 then goes down? WIll
search queries with preference value "user-1234" also fail from that point
onwards, or will it fail over, i.e., those custom preferences affected will
get re-mapped internally to other shards?

Thanks for any insights,
Patrick

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/219831d5-07e8-4764-af87-ff3d51b88bce%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Patrick Peschlow) #3

Hi Yarin,

when no one replied, I delved into the ElasticSearch source code to find
the answer myself. What I concluded from the code is that - assuming there
are N data nodes available in the cluster - the custom preference value is
transformed and eventually mapped to an index 0 .. N-1 via modulo. This
way, as long as the available nodes do not change, the queries will get
routed to the same node.

However, once a node leaves the cluster or another one joins, the index
resulting from the transformation may change and thus queries using the
same custom preference may get routed to another node. This implies that

  1. there is a failover of the custom preference mechanism, as long as there
    is at least one node available
  2. the preferred node may change whenever the set of available data nodes
    changes (not necessarily when that preferred node goes down but it can
    happen even when any node leaves or joins)

I hope that clarifies the issue for you. And if my understanding of the
code turns out to be wrong, I'll be glad to learn about it :slight_smile:

-Patrick

On Sunday, December 15, 2013 2:49:15 PM UTC+1, Yarin Miran wrote:

I'm also really interested in the answer for Patrick's question.

Bumping this in hope to get some answer.

Thanks in advance,
Yarin.

On Wednesday, August 28, 2013 2:59:14 PM UTC+3, Patrick Peschlow wrote:

The search preference documentation on
http://www.elasticsearch.org/guide/reference/api/search/preference/states that a custom (string) preference is supported. However, it does not
explicitly state how possible failovers are handled in that case. Suppose
that a custom preference value "user-1234" is currently (internally)
associated with shard 1. What happens if shard 1 then goes down? WIll
search queries with preference value "user-1234" also fail from that point
onwards, or will it fail over, i.e., those custom preferences affected will
get re-mapped internally to other shards?

Thanks for any insights,
Patrick

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a9cefab3-11c6-45e0-a6c9-ea047d5583df%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #4