# Does forcing/manually setting the id of the document makes the shards unevenly distributed?
**URL:** https://discuss.elastic.co/t/does-forcing-manually-setting-the-id-of-the-document-makes-the-shards-unevenly-distributed/317372
**Category:** Elasticsearch
**Created:** [October 25, 2022, 7:03am UTC](https://discuss.elastic.co/t/does-forcing-manually-setting-the-id-of-the-document-makes-the-shards-unevenly-distributed/317372 "2022-10-25T07:03:53Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@Java2avaj](https://discuss.elastic.co/u/Java2avaj)
#### Post date: [October 25, 2022, 7:03am UTC](https://discuss.elastic.co/t/does-forcing-manually-setting-the-id-of-the-document-makes-the-shards-unevenly-distributed/317372/1 "2022-10-25T07:03:53Z")
Does forcing/manually setting the id of the document makes the shards unevenly distributed?
1. Currently, we are forcing the values of id when storing documents in the index. Is this a bad practice? Does it make the shards unevenly distributed?
2. Does having elasticsearch autogenerate its own id makes the shards evenly distributed?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 25, 2022, 7:25am UTC](https://discuss.elastic.co/t/does-forcing-manually-setting-the-id-of-the-document-makes-the-shards-unevenly-distributed/317372/2 "2022-10-25T07:25:23Z")
> [@Java2avaj](#):
>
> Currently, we are forcing the values of id when storing documents in the index. Is this a bad practice? Does it make the shards unevenly distributed?
Using an external document ID can make updates and deletes more efficient as you do not need to search for the document as you know the ID. As every insert can potentially be an update it typically also slows down indexing throughput, especially as shards grow in size.
The shard is determined based on a hash of the document ID so a custom document ID scheme could result in unevenly balanced shards.
> [@Java2avaj](#):
>
> Does having elasticsearch autogenerate its own id makes the shards evenly distributed?
This typically results in quite evenly distributed shards.
---
### Author:  [@Java2avaj](https://discuss.elastic.co/u/Java2avaj)
#### Post date: [October 25, 2022, 7:57am UTC](https://discuss.elastic.co/t/does-forcing-manually-setting-the-id-of-the-document-makes-the-shards-unevenly-distributed/317372/3 "2022-10-25T07:57:06Z")
thanks for your response. Just a follow up question, is it possible to "re calculate" or re-assign documents to other shards to achieve an even distribution of data between the shards while still using an external document ID?
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 26, 2022, 1:18am UTC](https://discuss.elastic.co/t/does-forcing-manually-setting-the-id-of-the-document-makes-the-shards-unevenly-distributed/317372/4 "2022-10-26T01:18:23Z")
Only if you change the ID. Otherwise it'll allocate it to the same shard.
---
### Author:  [@Java2avaj](https://discuss.elastic.co/u/Java2avaj)
#### Post date: [October 27, 2022, 3:46am UTC](https://discuss.elastic.co/t/does-forcing-manually-setting-the-id-of-the-document-makes-the-shards-unevenly-distributed/317372/5 "2022-10-27T03:46:21Z")
I saw that the formula for determining which shard the data is stored is:
```auto
routing = _routing != null ? _routing : _id
routing_factor = num_routing_shards / num_primary_shards
shard_num = (hash(_routing) % num_routing_shards) / routing_factor
```
So that means, even if we manually force the id of the document, still, it will undergo to a hash function and will not be distributed on one shard only, thus also achieving an even distribution of shards. Is this correct?
In other words, even if the id is autogenerated by elasticsearch, still we cannot guarantee which shard will the document be placed because of hash function.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 27, 2022, 5:46am UTC](https://discuss.elastic.co/t/does-forcing-manually-setting-the-id-of-the-document-makes-the-shards-unevenly-distributed/317372/6 "2022-10-27T05:46:49Z")
You can use [routing](https://www.elastic.co/guide/en/elasticsearch/reference/8.4/mapping-routing-field.html) to override shard selection by document ID hash when indexing a document. This however means that you also will need to provide the same routing value every time you want to update or delete the document in the future. It does not necessarily solve the problem though as you can still get uneven distribution. The best way would probably be to improve the way you create document IDs.
What is the problem you are looking to solve? How uneven is your current distribution? What is your current document ID format?
---
### Author:  [@Java2avaj](https://discuss.elastic.co/u/Java2avaj)
#### Post date: [October 27, 2022, 6:13am UTC](https://discuss.elastic.co/t/does-forcing-manually-setting-the-id-of-the-document-makes-the-shards-unevenly-distributed/317372/7 "2022-10-27T06:13:07Z")
Since we will be applying this to all of our existing indices, the format of document id varies. Some are in string format, some are in auto incremented format.
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 29, 2022, 5:32am UTC](https://discuss.elastic.co/t/does-forcing-manually-setting-the-id-of-the-document-makes-the-shards-unevenly-distributed/317372/8 "2022-10-29T05:32:29Z")
Why not just use the default approach for that and make another custom field with your id value?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 26, 2022, 5:32am UTC](https://discuss.elastic.co/t/does-forcing-manually-setting-the-id-of-the-document-makes-the-shards-unevenly-distributed/317372/9 "2022-11-26T05:32:33Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.