# Help! I can't filter out docs with bad ids in reindex

**URL:** https://discuss.elastic.co/t/help-i-cant-filter-out-docs-with-bad-ids-in-reindex/280968
**Category:** Elasticsearch
**Created:** [August 10, 2021, 10:21pm UTC](https://discuss.elastic.co/t/help-i-cant-filter-out-docs-with-bad-ids-in-reindex/280968 "2021-08-10T22:21:03Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![tasslehawf](https://avatars.discourse-cdn.com/v4/letter/t/e19b73/32.png) [@tasslehawf](https://discuss.elastic.co/u/tasslehawf)
#### Post date: [August 10, 2021, 10:21pm UTC](https://discuss.elastic.co/t/help-i-cant-filter-out-docs-with-bad-ids-in-reindex/280968/1 "2021-08-10T22:21:03Z")

</div>

Our index has a number of docs with bad ids. They were set by this script during a previous reindex:

```auto
def lowerCaseAuthor = ctx.author_string != null ? ctx.author_string.toLowerCase() : null;
def lowerCasePub = ctx.publication_fqdn != null ? ctx.publication_fqdn.toLowerCase() : null;
ctx._id = lowerCaseAuthor + lowerCasePub + ctx.collection_id

```

The problem is that some our author\_string fields are very long. More than 256 characters long making the id fields longer than the 256 character limit.

Now that I am trying to reindex again, I am getting frequent errors trying to reindex.

```auto
"error": { - 
    "type": "action_request_validation_exception",
    "reason": "Validation Failed: 1: id is too long, must be no longer than 512 bytes but was: 529;"
  }

```

I had thought that maybe I could filter out the bad docs in the reindex. I came up with the query which seems to work fine as a standalone query:

```auto
{
    "query": {
    "bool" : {
        "filter": {
            "script": {
                "script":"doc['_id'].value.length() > 256"
            }
         }
        }
    }
}

```

To count the number of docs with ids that are too long. But when I apply this query script to my reindex:

```auto
{
  "source": {
    "index": "testing",
    "query": {
      "bool": {
        "filter": {
          "script": {
            "script": "doc['_id'].value.length() < 256"
          }
        }
      }
    }
  },
  "dest": {
    "index": "testing_v6"
  }
}

```

I am back to square one. It runs a bit and then I get the same validation error as above:

```auto
"error": { - 
    "type": "action_request_validation_exception",
    "reason": "Validation Failed: 1: id is too long, must be no longer than 512 bytes but was: 529;"
  }

```

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [August 10, 2021, 11:28pm UTC](https://discuss.elastic.co/t/help-i-cant-filter-out-docs-with-bad-ids-in-reindex/280968/2 "2021-08-10T23:28:40Z")

</div>

Please don't create multiple topics on the same thing - [Cannot reindex due to bad id field(s)](https://discuss.elastic.co/t/cannot-reindex-due-to-bad-id-field-s/280601)

Feel free to bump your original topic and add a bit more info to it though 🙂

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [August 10, 2021, 11:28pm UTC](https://discuss.elastic.co/t/help-i-cant-filter-out-docs-with-bad-ids-in-reindex/280968/3 "2021-08-10T23:28:43Z")

</div>


