# Cannot reindex due to bad id field(s)

**URL:** https://discuss.elastic.co/t/cannot-reindex-due-to-bad-id-field-s/280601
**Category:** Elasticsearch
**Tags:** painless
**Created:** [August 5, 2021, 9:46pm UTC](https://discuss.elastic.co/t/cannot-reindex-due-to-bad-id-field-s/280601 "2021-08-05T21:46:01Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![tasslehawf](https://avatars.discourse-cdn.com/v4/letter/t/e19b73/32.png) [@tasslehawf](https://discuss.elastic.co/u/tasslehawf)
#### Post date: [August 5, 2021, 9:46pm UTC](https://discuss.elastic.co/t/cannot-reindex-due-to-bad-id-field-s/280601/1 "2021-08-05T21:46:01Z")

</div>

I ran into an issue where we are trying to reindex in order to gain a copy\_to search field.

```auto
"type": "action_request_validation_exception",
    "reason": "Validation Failed: 1: id is too long, must be no longer than 512 bytes but was: 877;"

```

During the last reindex, ids were written that were longer than 512 bytes.

I was hoping that we could target these particular bad \_ids with this script:

```auto
{
    "script": {
"source": "if (ctx._id.length() > 512) {ctx._id = 'null'}",
"lang": "painless"
},
  "source": {
    "index": "testing"
  },
  "dest": {
    "index": "testing_v5"
  }
}

```

But I still get the same error as above.

Also I noticed then when I do a large reindex, with the `?wait_for_completion=false` even after the task says its complete, it seems to take a long time before docs are available.

---

<div class="post-metadata">

### Author: ![tasslehawf](https://avatars.discourse-cdn.com/v4/letter/t/e19b73/32.png) [@tasslehawf](https://discuss.elastic.co/u/tasslehawf)
#### Post date: [August 11, 2021, 12:05am UTC](https://discuss.elastic.co/t/cannot-reindex-due-to-bad-id-field-s/280601/2 "2021-08-11T00:05:52Z")

</div>

Here are my latest issues:

Our index has a number of docs with bad ids. They were set by this script during a previous reindex:

```auto
def lowerCaseAuthor = ctx.author_string != null ? ctx.author_string.toLowerCase() : null;
def lowerCasePub = ctx.publication_fqdn != null ? ctx.publication_fqdn.toLowerCase() : null;
ctx._id = lowerCaseAuthor + lowerCasePub + ctx.collection_id

```

The problem is that some our author\_string fields are very long. More than 256 characters long making the id fields longer than the 256 character limit.

Now that I am trying to reindex again, I am getting frequent errors trying to reindex.

```auto
"error": { - 
    "type": "action_request_validation_exception",
    "reason": "Validation Failed: 1: id is too long, must be no longer than 512 bytes but was: 529;"
  }

```

I had thought that maybe I could filter out the bad docs in the reindex. I came up with the query which seems to work fine as a standalone query:

```auto
{
    "query": {
    "bool" : {
        "filter": {
            "script": {
                "script":"doc['_id'].value.length() > 256"
            }
         }
        }
    }
}

```

To count the number of docs with ids that are too long. But when I apply this query script to my reindex:

```auto
{
  "source": {
    "index": "testing",
    "query": {
      "bool": {
        "filter": {
          "script": {
            "script": "doc['_id'].value.length() < 256"
          }
        }
      }
    }
  },
  "dest": {
    "index": "testing_v6"
  }
}

```

I am back to square one. It runs a bit and then I get the same validation error as above:

```auto
"error": { - 
    "type": "action_request_validation_exception",
    "reason": "Validation Failed: 1: id is too long, must be no longer than 512 bytes but was: 529;"
  }

```

---

<div class="post-metadata">

### Author: ![tasslehawf](https://avatars.discourse-cdn.com/v4/letter/t/e19b73/32.png) [@tasslehawf](https://discuss.elastic.co/u/tasslehawf)
#### Post date: [August 20, 2021, 6:45pm UTC](https://discuss.elastic.co/t/cannot-reindex-due-to-bad-id-field-s/280601/3 "2021-08-20T18:45:06Z")

</div>

The solution is that the id field is limited to 128 CHARACTERS.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 17, 2021, 6:45pm UTC](https://discuss.elastic.co/t/cannot-reindex-due-to-bad-id-field-s/280601/4 "2021-09-17T18:45:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.