# Performance/Constraints on provided external document id

**URL:** <https://discuss.elastic.co/t/performance-constraints-on-provided-external-document-id/26469>\
**Category:** Elasticsearch\
**Created:** [July 29, 2015, 9:00am UTC](https://discuss.elastic.co/t/performance-constraints-on-provided-external-document-id/26469 "2015-07-29T09:00:04Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![soltiz](https://avatars.discourse-cdn.com/v4/letter/s/df705f/32.png) [@soltiz](https://discuss.elastic.co/u/soltiz)\
**Post date:** [July 29, 2015, 9:00am UTC](https://discuss.elastic.co/t/performance-constraints-on-provided-external-document-id/26469/1 "2015-07-29T09:00:04Z")

</div>

Hi everyone  
For deduplication purpose, we would like to provide our own id when inserting/updating documents.  
We do not need versioning of documents. Sender always do "UPSERTS" (i.e. it doesn't know if document has already been indexed).  
Question is multifold :

- Is there performance impact when providing an external ID (because ES will not now if it is a create/update) ?
- Will providing target version (always "first version") of the document help (because ES will not have to find out a new version available to store document) ?
- Are there advices on the form of the external ID ( max size, sharding algorithm...) that affect performance/storage space ?

Best regards

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [July 29, 2015, 12:07pm UTC](https://discuss.elastic.co/t/performance-constraints-on-provided-external-document-id/26469/2 "2015-07-29T12:07:03Z")

</div>

> [@soltiz](#):
>
> - Are there advices on the form of the external ID ( max size, sharding algorithm...) that affect performance/storage space ?

> **[Choosing a fast unique identifier (UUID) for Lucene](https://blog.mikemccandless.com/2014/05/choosing-fast-unique-identifier-uuid.html)**
>
> Most search applications using Apache Lucene assign a unique id, or primary key, to each indexed document. While Lucene itself does not...

That suggests that zero padded sequential or mostly sequential ids does best. I've done just fine with sequential ids just strait from the database.

> [@soltiz](#):
>
> - Will providing target version (always "first version") of the document help (because ES will not have to find out a new version available to store document) ?

That'll break things I think. Do not provide a version if you always want Elasticsearch to perform the update.

> [@soltiz](#):
>
> - Is there performance impact when providing an external ID (because ES will not now if it is a create/update) ?

You can provide an external id with a index request rather than an update request. I believe it to be slightly faster to send index requests if you know that the document is new but its not tons and tons faster and its harder to get right in multithreaded upserting environments. I've done fine by always sending updates.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:58pm UTC](https://discuss.elastic.co/t/performance-constraints-on-provided-external-document-id/26469/3 "2017-07-05T23:58:22Z")

</div>