Reindexing failed - Validation Failed: 1: id is too long, must be no longer than 512 bytes but was: 672;

I am reindexing the elasticsearch with the cluster restart from 2.4.1 to 5.6.16, I have reindexed few indices successfull. Out of 23 indices 3 are getting failed duw to the Validation Failure with the id.

Validation Failed: 1: id is too long, must be no longer than 512 bytes but was: 672;

Validation Failed: 1: id is too long, must be no longer than 512 bytes but was: 549;2: id is too long, must be no longer than 512 bytes but was: 517;3: id is too long, must be no longer than 512bytes but was: 532;4: id is too long, must be no longer than 512 bytes but was: 719;5: id is too long, must be no longer than 512 bytes but was: 1374;6: id is too long, must be no longer than 512 bytes but was:614;7: id is too long, must be no longer than 512 bytes but was: 623;8: id is too long, must be no longer than 512 bytes but was: 543;9: id is too long, must be no longer than 512 bytes but was: 895;10: id is too long, must be no longer than 512 bytes but was: 697;11: id is too long, must be no longer than 512 bytes but was: 1064;12: id is too long, must be no longer than 512 bytes but was: 785;13: id is too long, must be no longer than 512 bytes but was: 600;14: id is too long, must be no longer than 512 bytes but was: 537;15: id is too long, must be no longer than 512 bytes but was: 870;16: id is too long, must be no longer than 512 bytes but was: 802;17: id is too long, must be no longer than 512 bytes but was: 716;18: id is too long, must be no longer than 512 bytes but was: 844;19: id is too long, must be no longer than 512 bytes but was: 1503;20: id is too long, must be no longer than 512 bytes but was: 734;21: id is too long, must be no longer than 512 bytes but was: 590;22: id is too long, must be no longer than 512 bytes but was:697;23: id is too long, must be no longer than 512 bytes but was: 1008;24: id is too long, must be no longer than 512 bytes but was: 1146;25: id is too long, must be no longer than 512 bytes but was: 609;26: idis too long, must be no longer than 512 bytes but was: 988;27: id is too long, must be no longer than 512 bytes but was: 703;28: id is too long, must be no longer than 512 bytes but was: 568;29: id is too long,must be no longer than 512 bytes but was: 726;30: id is too long, must be no longer than 512 bytes but was: 737;31: id is too long, must be no longer than 512 bytes but was: 611;32: id is too long, must be no longer than 512 bytes but was: 914;

This limit is hard-coded and can't be changed, so you'll need to find a way to reduce the length of the ID field.
A similar issue came up recently and it was noted that the limit isn't documented anywhere. I think the docs are in the process of being updated.

1 Like

Doing it manually is very infinite steps to identify the records, i have same data in other database along with elasticsearch.

Elasticsearch team may need to suggest something.