# Understanding "id existence" check mechanism for custom ids **URL:** \ **Category:** Elasticsearch\ **Created:** [June 2, 2025, 1:32pm UTC](https://discuss.elastic.co/t/understanding-id-existence-check-mechanism-for-custom-ids/378795 "2025-06-02T13:32:43Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![danslapman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danslapman/32/143485_2.png) [@danslapman](https://discuss.elastic.co/u/danslapman)\ **Post date:** [June 2, 2025, 1:32pm UTC](https://discuss.elastic.co/t/understanding-id-existence-check-mechanism-for-custom-ids/378795/1 "2025-06-02T13:32:44Z")
Greetings! I'm seeking a way to verify/clarify my understanding on what's going on during indexing a document with custom \_id. I know that ES performs an existence check in such case, but I'm curious about details. For my understanding, ES does the following: 1. detetrmine shard (by hashing id or provided routing key) where document is potentially stored 2. perform a lookup within that shard So, roughly my question is "there is no full index scan in such case, right"? Thanks in advance 🙏 ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 2, 2025, 1:41pm UTC](https://discuss.elastic.co/t/understanding-id-existence-check-mechanism-for-custom-ids/378795/2 "2025-06-02T13:41:10Z")
That does sound correct. Only the shard the document is being indexed into is searched. ---
**Author:** ![danslapman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danslapman/32/143485_2.png) [@danslapman](https://discuss.elastic.co/u/danslapman)\ **Post date:** [June 2, 2025, 1:54pm UTC](https://discuss.elastic.co/t/understanding-id-existence-check-mechanism-for-custom-ids/378795/3 "2025-06-02T13:54:22Z")
Thank you very much for you reply! ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [June 2, 2025, 3:03pm UTC](https://discuss.elastic.co/t/understanding-id-existence-check-mechanism-for-custom-ids/378795/4 "2025-06-02T15:03:39Z")
> [@danslapman](#): > > there is no full index scan There is not even anything you could call a "full index scan" within the single shard that owns the document ID. It's just checking the terms dictionary -- think something like a B-tree -- so it's only logarithmic effort. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [June 2, 2025, 5:50pm UTC](https://discuss.elastic.co/t/understanding-id-existence-check-mechanism-for-custom-ids/378795/5 "2025-06-02T17:50:30Z")
Hi @danslapman At the risk of being "corrected" by @DavidTurner🙂 Note: High-throughput ingest use cases with self-generated IDs with larger shard sizes might negatively affect performance. While using your own `_id` provides flexibility, it could impact ingest efficiency.... so it is "not free" but as efficient as possible. The "unusual" part of this is that you might get "uneven" ingest pattern when the shards are brand new ... little to no impact .... when large shards are reaching their limit the impact can become larger... then roll over happens and the pattern resets... ---
**Author:** ![danslapman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danslapman/32/143485_2.png) [@danslapman](https://discuss.elastic.co/u/danslapman)\ **Post date:** [June 2, 2025, 6:05pm UTC](https://discuss.elastic.co/t/understanding-id-existence-check-mechanism-for-custom-ids/378795/6 "2025-06-02T18:05:47Z")
Hi @stephenb Thanks for the explanation! Yes, I understand that auto-generated ids are more efficient. In my case I have a system that generates ids on application side already and I wanted to check/verify my understanding before I do some optimization steps (we cannot switch to auto-generation quickly)