# How does Elasticsearch map Integer doc IDs to shards
**URL:** https://discuss.elastic.co/t/how-does-elasticsearch-map-integer-doc-ids-to-shards/261329
**Category:** Elasticsearch
**Created:** [January 16, 2021, 6:50pm UTC](https://discuss.elastic.co/t/how-does-elasticsearch-map-integer-doc-ids-to-shards/261329 "2021-01-16T18:50:36Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@rex-remind](https://discuss.elastic.co/u/rex-remind)
#### Post date: [January 16, 2021, 6:50pm UTC](https://discuss.elastic.co/t/how-does-elasticsearch-map-integer-doc-ids-to-shards/261329/1 "2021-01-16T18:50:37Z")
How does Elasticsearch map Integer doc IDs to shards? What algorithm does it use?
I've been digging around but haven't seemed to find the answer.
For context, every document we have has an id that's a row id from postgres. We're seeing skew on our nodes, likely because many rows have been deleted at different id ranges of this table in the past and if it's simply doing a modulo the documents may not evenly distribute.
Thanks
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [January 16, 2021, 11:16pm UTC](https://discuss.elastic.co/t/how-does-elasticsearch-map-integer-doc-ids-to-shards/261329/2 "2021-01-16T23:16:46Z")
Do the responses in this topic help? [What algorithm is ElasticSearch create Document \_Id based on?Could somebody answer me，plz](https://discuss.elastic.co/t/what-algorithm-is-elasticsearch-create-document-id-based-on-could-somebody-answer-me-plz/166317)
---
### Author:  [@rex-remind](https://discuss.elastic.co/u/rex-remind)
#### Post date: [January 17, 2021, 1:42am UTC](https://discuss.elastic.co/t/how-does-elasticsearch-map-integer-doc-ids-to-shards/261329/3 "2021-01-17T01:42:24Z")
Afaiu that post is for auto-generated `_id`s. Our integer `id`s from postgres are used as the id for the document, which seems to be the same as `_id` therefore we don't have auto-generated ids.
I'm wondering how these Integer `id`s/`_id`s will map to a shard.
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [January 17, 2021, 3:14am UTC](https://discuss.elastic.co/t/how-does-elasticsearch-map-integer-doc-ids-to-shards/261329/4 "2021-01-17T03:14:18Z")
Look [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-routing-field.html)
> A document is routed to a particular shard in an index using the following formula:
>
> shard\_num = hash(\_routing) % num\_primary\_shards
>
> The default value used for `_routing` is the document’s [`_id`](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-id-field.html).
So in your case you are using your row number as` _id` if I understand
As to the exact hash function you would need to look at the code, there are a lot of reasons that nodes / shards can skew over time, the deletions you spoke could be part of it. You can reindex etc if it is really causing problems etc...
---
### Author:  [@rex-remind](https://discuss.elastic.co/u/rex-remind)
#### Post date: [January 17, 2021, 3:26am UTC](https://discuss.elastic.co/t/how-does-elasticsearch-map-integer-doc-ids-to-shards/261329/5 "2021-01-17T03:26:06Z")
What is the `hash` function?
We've had this indexing for a week so no go on reindexing, we'll just end up where we left off.
We turned off our job though and after waiting some time the shards taking more storage balanced out in size much closer to the same size as the rest of the shards, this makes me think that it's not documents that are unbalanced, but which documents are being updated.
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [January 17, 2021, 4:27am UTC](https://discuss.elastic.co/t/how-does-elasticsearch-map-integer-doc-ids-to-shards/261329/6 "2021-01-17T04:27:56Z")
Perhaps take a look at [this] ([https://www.elastic.co/blog/efficient-duplicate-prevention-for-event-based-data-in-elasticsearch](https://www.elastic.co/blog/efficient-duplicate-prevention-for-event-based-data-in-elasticsearch))
It talks about the concepts.
As to the actual hash function you will need to look it up in the code it's all open.
It could be this one but I am not positive
>
Perhaps autogenerated IDs might be a better solution and keep the row id as a term for quick lookup.
---
### Author:  [@rex-remind](https://discuss.elastic.co/u/rex-remind)
#### Post date: [January 17, 2021, 5:14am UTC](https://discuss.elastic.co/t/how-does-elasticsearch-map-integer-doc-ids-to-shards/261329/7 "2021-01-17T05:14:50Z")
We have to bulk update documents at a high pace. If the ID is autogenerated then how do I tell Elasticsearch what documents to replace?
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [January 17, 2021, 6:58pm UTC](https://discuss.elastic.co/t/how-does-elasticsearch-map-integer-doc-ids-to-shards/261329/8 "2021-01-17T18:58:59Z")
The are design tradeoffs in every system if you need to use the \_id for update perhaps you will need to figure out another way to balance the shards... It's hard to say without knowing all your requirement (which I am not asking for 😉 )
Perhaps your [postgresql could generate uuids](https://www.postgresql.org/docs/current/uuid-ossp.html)
Updates often need to be carefully considered.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 14, 2021, 6:59pm UTC](https://discuss.elastic.co/t/how-does-elasticsearch-map-integer-doc-ids-to-shards/261329/9 "2021-02-14T18:59:27Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.