Hashing algo. for Routing

Elastic Stack Elasticsearch
1

Hi,

I wanted to know which hashing algorithm is used by ES for sharding. I can
find 2 in
https://github.com/elasticsearch/elasticsearch/tree/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash.

Also, is there a way of specifying a custom hashing algorithm? The problem
I'm trying to solve is that the no. of routing keys that I have is around 2
times the number of shards in the index and I want each shard to support
two routing keys.

Thanks,
Anand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hi,
have a look at OperationRoutingModulehttps://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/OperationRoutingModule.javato find out more. That's the guice module that loads by default
the DjbHashFunction. Never tried it myself but it looks like it's also
possible to plug in a custom hash function that implements the HashFunction
interface, using the cluster.routing.operation.hash.type setting.

Hope this helps
Luca

On Monday, September 2, 2013 9:41:11 AM UTC+2, Anand Nalya wrote:

Hi,

I wanted to know which hashing algorithm is used by ES for sharding. I can
find 2 in
https://github.com/elasticsearch/elasticsearch/tree/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash
.

Also, is there a way of specifying a custom hashing algorithm? The problem
I'm trying to solve is that the no. of routing keys that I have is around 2
times the number of shards in the index and I want each shard to support
two routing keys.

Thanks,
Anand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi Anand,
any news? Looking back at your question, I wonder why you need to write a
custom hashing algorithm. Can't you just use custom routing? Would love to
know more about your usecase.

Cheers
Luca

On Monday, September 2, 2013 6:06:51 PM UTC+2, Luca Cavanna wrote:

Hi,
have a look at OperationRoutingModulehttps://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/OperationRoutingModule.javato find out more. That's the guice module that loads by default
the DjbHashFunction. Never tried it myself but it looks like it's also
possible to plug in a custom hash function that implements the HashFunction
interface, using the cluster.routing.operation.hash.type setting.

Hope this helps
Luca

On Monday, September 2, 2013 9:41:11 AM UTC+2, Anand Nalya wrote:

Hi,

I wanted to know which hashing algorithm is used by ES for sharding. I
can find 2 in
https://github.com/elasticsearch/elasticsearch/tree/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash
.

Also, is there a way of specifying a custom hashing algorithm? The
problem I'm trying to solve is that the no. of routing keys that I have is
around 2 times the number of shards in the index and I want each shard to
support two routing keys.

Thanks,
Anand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Hi Luca,

My use case was to have completely predictable routing of documents at
shard level. For example, I wanted to have the document with particular
value of attribute a1 and a2 to go to shard0 and for some other combination
to shard2.

To solve this, I did use a custom routing algorithm that gives a routing
value from A to (A+noOfShards-1). In this way even though I cannot
pinpoint the actual shard where the data goes, I can be sure that data
with routing key k and k+1 goes to adjacent shards.

Anand

On Wednesday, 11 September 2013 19:46:12 UTC+5:30, Luca Cavanna wrote:

Hi Anand,
any news? Looking back at your question, I wonder why you need to write a
custom hashing algorithm. Can't you just use custom routing? Would love to
know more about your usecase.

Cheers
Luca

On Monday, September 2, 2013 6:06:51 PM UTC+2, Luca Cavanna wrote:

Hi,
have a look at OperationRoutingModulehttps://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/OperationRoutingModule.javato find out more. That's the guice module that loads by default
the DjbHashFunction. Never tried it myself but it looks like it's also
possible to plug in a custom hash function that implements the HashFunction
interface, using the cluster.routing.operation.hash.type setting.

Hope this helps
Luca

On Monday, September 2, 2013 9:41:11 AM UTC+2, Anand Nalya wrote:

Hi,

I wanted to know which hashing algorithm is used by ES for sharding. I
can find 2 in
https://github.com/elasticsearch/elasticsearch/tree/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash
.

Also, is there a way of specifying a custom hashing algorithm? The
problem I'm trying to solve is that the no. of routing keys that I have is
around 2 times the number of shards in the index and I want each shard to
support two routing keys.

Thanks,
Anand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.