# Sharding Function **URL:** \ **Category:** Elasticsearch\ **Created:** [November 25, 2012, 8:59pm UTC](https://discuss.elastic.co/t/sharding-function/9818 "2012-11-25T20:59:15Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Roman\_Kournjaev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/roman_kournjaev/32/2552_2.png) [@Roman\_Kournjaev](https://discuss.elastic.co/u/Roman_Kournjaev)\ **Post date:** [November 25, 2012, 8:59pm UTC](https://discuss.elastic.co/t/sharding-function/9818/1 "2012-11-25T20:59:15Z")
Hi I am trying to optimize my indexing process ( in terms of time ). I am running a 8 shard cluster ( no replicas ) on 4 nodes. ( 4 physical machines ). I am indexing some set of 30M products , and I want actually to opimize the process of idexing that i send a specific subset of products to the exact node where it is going to be stored. Whats the sharding function elastic is using , and can i actually change it ? If yes , is that actually going to speed up my indexing process ? Thanks Roman -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 25, 2012, 9:45pm UTC](https://discuss.elastic.co/t/sharding-function/9818/2 "2012-11-25T21:45:52Z")
First of all, you can simply use NodeClient and it will do this for you. But if you feel really adventurous 🙂 you can replace DjbHashFunction[https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash/djb/DjbHashFunction.java](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash/djb/DjbHashFunction.java)that ES is using by implementing HashFunction[https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash/HashFunction.java](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash/HashFunction.java)interface and specifying your function's class name in the cluster.routing.operation.hash.type setting. It might speed up your indexing process if it's network bound. However, that would be rather a non-typical situation since in most cases indexing is CPU or disk I/O bound activity. If you have more than 2 processor cores on your nodes and indexing is not disk I/O bound, I would start with increasing the number of shards first. On Sunday, November 25, 2012 3:59:15 PM UTC-5, Roman Kournjaev wrote: > Hi > > I am trying to optimize my indexing process ( in terms of time ). > I am running a 8 shard cluster ( no replicas ) on 4 nodes. ( 4 physical > machines ). > > I am indexing some set of 30M products , and I want actually to opimize > the process of idexing that i send a specific subset of products to the > exact node where it is going to be stored. Whats the sharding function > elastic is using , and can i actually change it ? > > If yes , is that actually going to speed up my indexing process ? > > Thanks > Roman -- ---
**Author:** ![Roman\_Kournjaev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/roman_kournjaev/32/2552_2.png) [@Roman\_Kournjaev](https://discuss.elastic.co/u/Roman_Kournjaev)\ **Post date:** [November 25, 2012, 10:13pm UTC](https://discuss.elastic.co/t/sharding-function/9818/3 "2012-11-25T22:13:01Z")
I am accessing ES through Nest (C#) , so i probably dont have access not to the NodeClient. Can i provide this HashFunction through a http request ? On Sunday, November 25, 2012 11:45:52 PM UTC+2, Igor Motov wrote: > First of all, you can simply use NodeClient and it will do this for you. > But if you feel really adventurous 🙂 you can replace DjbHashFunction[https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash/djb/DjbHashFunction.java](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash/djb/DjbHashFunction.java)that ES is using by implementing > HashFunction[https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash/HashFunction.java](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash/HashFunction.java)interface and specifying your function's class name in the > cluster.routing.operation.hash.type setting. It might speed up your > indexing process if it's network bound. However, that would be rather a > non-typical situation since in most cases indexing is CPU or disk I/O bound > activity. If you have more than 2 processor cores on your nodes and > indexing is not disk I/O bound, I would start with increasing the number of > shards first. > > On Sunday, November 25, 2012 3:59:15 PM UTC-5, Roman Kournjaev wrote: > > > Hi > > > > I am trying to optimize my indexing process ( in terms of time ). > > I am running a 8 shard cluster ( no replicas ) on 4 nodes. ( 4 physical > > machines ). > > > > I am indexing some set of 30M products , and I want actually to opimize > > the process of idexing that i send a specific subset of products to the > > exact node where it is going to be stored. Whats the sharding function > > elastic is using , and can i actually change it ? > > > > If yes , is that actually going to speed up my indexing process ? > > > > Thanks > > Roman -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 25, 2012, 10:26pm UTC](https://discuss.elastic.co/t/sharding-function/9818/4 "2012-11-25T22:26:06Z")
No, the function should be implemented in form of elasticsearch plugin. On Sunday, November 25, 2012 5:13:01 PM UTC-5, Roman Kournjaev wrote: > I am accessing ES through Nest (C#) , so i probably dont have access not > to the NodeClient. Can i provide this HashFunction through a http request ? > > On Sunday, November 25, 2012 11:45:52 PM UTC+2, Igor Motov wrote: > > > First of all, you can simply use NodeClient and it will do this for you. > > But if you feel really adventurous 🙂 you can replace DjbHashFunction[https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash/djb/DjbHashFunction.java](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash/djb/DjbHashFunction.java)that ES is using by implementing > > HashFunction[https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash/HashFunction.java](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/operation/hash/HashFunction.java)interface and specifying your function's class name in the > > cluster.routing.operation.hash.type setting. It might speed up your > > indexing process if it's network bound. However, that would be rather a > > non-typical situation since in most cases indexing is CPU or disk I/O bound > > activity. If you have more than 2 processor cores on your nodes and > > indexing is not disk I/O bound, I would start with increasing the number of > > shards first. > > > > On Sunday, November 25, 2012 3:59:15 PM UTC-5, Roman Kournjaev wrote: > > > > > Hi > > > > > > I am trying to optimize my indexing process ( in terms of time ). > > > I am running a 8 shard cluster ( no replicas ) on 4 nodes. ( 4 physical > > > machines ). > > > > > > I am indexing some set of 30M products , and I want actually to opimize > > > the process of idexing that i send a specific subset of products to the > > > exact node where it is going to be stored. Whats the sharding function > > > elastic is using , and can i actually change it ? > > > > > > If yes , is that actually going to speed up my indexing process ? > > > > > > Thanks > > > Roman -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:03am UTC](https://discuss.elastic.co/t/sharding-function/9818/5 "2017-07-06T03:03:00Z")
Closed on July 6, 2017, 3:03am UTC