# \_id length

**URL:** <https://discuss.elastic.co/t/id-length/182245>\
**Category:** Elasticsearch\
**Created:** [May 22, 2019, 2:12pm UTC](https://discuss.elastic.co/t/id-length/182245 "2019-05-22T14:12:09Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![jbalaguero](https://avatars.discourse-cdn.com/v4/letter/j/c68b51/32.png) [@jbalaguero](https://discuss.elastic.co/u/jbalaguero)\
**Post date:** [May 22, 2019, 2:12pm UTC](https://discuss.elastic.co/t/id-length/182245/1 "2019-05-22T14:12:09Z")

</div>

We are using ES6.4 to index a cache whose values are xml documents. The key of this cache is generated capturing values from the xml document, so it's easy to get ids longer than 512 bytes.

Then when we perform an index request, the \_id must be this key and since it's longer than 512, the bulk request fails.

So, is it possible (by config or in any way) to remove this limitation?

---

<div class="post-metadata">

**Author:** ![whatgeorgemade](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/whatgeorgemade/32/103246_2.png) [@whatgeorgemade](https://discuss.elastic.co/u/whatgeorgemade)\
**Post date:** [May 22, 2019, 2:56pm UTC](https://discuss.elastic.co/t/id-length/182245/2 "2019-05-22T14:56:40Z")

</div>

I wasn't even aware of a length restriction on the `_id` field. There's nothing about it in [the docs](https://www.elastic.co/guide/en/elasticsearch/reference/master/mapping-id-field.html) so I'm not sure if it can be removed/changed.

Do you have to use that value as the actual document ID? If not, you could let Elasticsearch generate an `_id` value for you (by not specifying one yourself), then add your own ID value to a different field that's mapped as a `keyword` type. It would have implications on how you access the documents but those may not affect you.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 22, 2019, 3:20pm UTC](https://discuss.elastic.co/t/id-length/182245/3 "2019-05-22T15:20:26Z")

</div>

There's a 512 byte limit, you may want to hash it down to that.

I'll raise an issue to get that documented 🙂

---

<div class="post-metadata">

**Author:** ![jbalaguero](https://avatars.discourse-cdn.com/v4/letter/j/c68b51/32.png) [@jbalaguero](https://discuss.elastic.co/u/jbalaguero)\
**Post date:** [May 22, 2019, 3:53pm UTC](https://discuss.elastic.co/t/id-length/182245/4 "2019-05-22T15:53:35Z")

</div>

Our development is external, here I simplified the question but it's a bit more complicated. This cache belongs to a bank ... This hash you suggest implies work not only on our side but on their side... I don't think they are going to change the cache design because I tell them ES does not support ids longer than 512 bytes.

Instead of this, why not to add a new config parameter, something like "index.max\_id\_length=xxxx", and set it to 512 by default?

Thanks.

And when you validate the id length, instead of "id\_length \> 512 then error" write "id\_length \> index.max\_id\_length then error".

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 22, 2019, 10:51pm UTC](https://discuss.elastic.co/t/id-length/182245/5 "2019-05-22T22:51:33Z")

</div>

Can you potentially look at moving that value _out_ of `_id`?

---

<div class="post-metadata">

**Author:** ![jbalaguero](https://avatars.discourse-cdn.com/v4/letter/j/c68b51/32.png) [@jbalaguero](https://discuss.elastic.co/u/jbalaguero)\
**Post date:** [May 23, 2019, 7:32am UTC](https://discuss.elastic.co/t/id-length/182245/6 "2019-05-23T07:32:04Z")

</div>

We need to remove documents using the \_id. In the remove method we only receive the cacheId and the cache object. If we use an autogenerated ID when inserting documents, we will need to store it into the cache object, and now that's not possible because we don't control this object. So my only chance is to add an intermediate concurrentmap where the key is the cacheId and the value the autogenerated id. Yes, I can do it.

But my question is: is this 512 bytes limitation a technical limitation? So if you say it is technically impossible to have ids longer than 512 then ok, that's the end of the story.

But if this limitation is because at any point in the past someone though that having such a long ids would be awful in terms of performance, maybe it would be great to let this decission to the end user. If someone needs longer ids, and if this is a performance problem it will be his/her job to request for more cores, memory or whatever he/she needs.

I don't know what implications can have to allow these longer ids (by config) in terms of your code. Maybe it will be easy to do it if this limit is only used in validation time when indexing documents, but I don't know if it's being used for something else.

Anyways, thanks for your time 🙂

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 23, 2019, 10:13pm UTC](https://discuss.elastic.co/t/id-length/182245/7 "2019-05-23T22:13:40Z")

</div>

Doc PR at [Update id-field.asciidoc by markwalkom · Pull Request #42482 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/pull/42482)

> [@jbalaguero](#):
>
> But my question is: is this 512 bytes limitation a technical limitation? So if you say it is technically impossible to have ids longer than 512 then ok, that's the end of the story.

It's a technical limitation in that it's a coded one. I don't know why it's there though, you may want to raise an issue to seek further clarification.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 20, 2019, 10:13pm UTC](https://discuss.elastic.co/t/id-length/182245/8 "2019-06-20T22:13:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
