# Elasticsearch version upgrade issue -- CorruptIndexException **URL:** \ **Category:** Elasticsearch\ **Created:** [September 9, 2014, 11:18pm UTC](https://discuss.elastic.co/t/elasticsearch-version-upgrade-issue-corruptindexexception/19696 "2014-09-09T23:18:53Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Wei](https://avatars.discourse-cdn.com/v4/letter/w/90ced4/32.png) [@Wei](https://discuss.elastic.co/u/Wei)\ **Post date:** [September 9, 2014, 11:18pm UTC](https://discuss.elastic.co/t/elasticsearch-version-upgrade-issue-corruptindexexception/19696/1 "2014-09-09T23:18:53Z")
Hi All, I'm working on an ES upgrade from v0.20.5 to v1.2.1 I tested in a 2 node cluster, 3 indices, ~4 million docs, 18G file sizes, 20 shards, 1 replicas However, after bumping the version and reboot the cluster, I kept on seeing some shards are damaged. The ES log said: Caused by: org.apache.lucene.index.CorruptIndexException: did not read all bytes from file: read 451 vs size 452 (resource: BufferedChecksumIndexInput(MMapIndexInput(path="/18/index/\_195c\_i.del"))) This badly blocked the version upgrade in my case. Could you any one point me the reason of this issue? Lots appreciate to your help! Wei -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e315ae9e-5c4a-43ab-a48f-3201dc52c6c9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e315ae9e-5c4a-43ab-a48f-3201dc52c6c9%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Scott\_Decker](https://avatars.discourse-cdn.com/v4/letter/s/7bcc69/32.png) [@Scott\_Decker](https://discuss.elastic.co/u/Scott_Decker)\ **Post date:** [September 10, 2014, 2:01pm UTC](https://discuss.elastic.co/t/elasticsearch-version-upgrade-issue-corruptindexexception/19696/2 "2014-09-10T14:01:47Z")
We did not run into this issue when we upgraded from 20.6 to 1.3.1, but from looking at the upgrade docs, we did a few things to try and protect any index corruption, which looks like what you ran into. 1 - we stopped any apps from writing to the indexes when we started our upgrade 2 - we flushed the cluster before bringing it down 3 - we disabled shard allocation/replication before bringing the cluster down (just to make sure all nodes brought back the indexes that were on the same machines.) 4 - when we brought everything back up, we ran optimize on each index. This was noted as a task to do, because all of the indexes are in new formats in the newer releases, so, it was recommend to run the optimize which would recreate all of the indexes. It was unclear whether older indexes would really work in upgraded cluster. we did not take the chance and took the time hit to run optimize. 5 - re-enabled shard/replication allocations 6 - cluster was working just fine hope our steps help you try and redo the cluster upgrade. On Tuesday, September 9, 2014 4:18:54 PM UTC-7, Wei wrote: > Hi All, > > I'm working on an ES upgrade from v0.20.5 to v1.2.1 > I tested in a 2 node cluster, 3 indices, ~4 million docs, 18G file sizes, > 20 shards, 1 replicas > However, after bumping the version and reboot the cluster, I kept on > seeing some shards are damaged. The ES log said: > Caused by: org.apache.lucene.index.CorruptIndexException: did not read all > bytes from file: read 451 vs size 452 (resource: > BufferedChecksumIndexInput(MMapIndexInput(path="/18/index/\_195c\_i.del"))) > > This badly blocked the version upgrade in my case. > Could you any one point me the reason of this issue? > Lots appreciate to your help! > > Wei -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/cb341c55-2a15-465e-81ae-463ffb523355%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cb341c55-2a15-465e-81ae-463ffb523355%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Robert\_Muir\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robert_muir_2/32/931_2.png) [@Robert\_Muir\_2](https://discuss.elastic.co/u/Robert_Muir_2)\ **Post date:** [September 23, 2014, 11:24pm UTC](https://discuss.elastic.co/t/elasticsearch-version-upgrade-issue-corruptindexexception/19696/3 "2014-09-23T23:24:40Z")
This is a bug in lucene: [[LUCENE-5975] Lucene can't read 3.0-3.3 deleted documents - ASF JIRA](https://issues.apache.org/jira/browse/LUCENE-5975) Sorry it took a while, thanks for reporting this! On Tue, Sep 9, 2014 at 7:18 PM, Wei [wshen@groupon.com](mailto:wshen@groupon.com) wrote: > Hi All, > > I'm working on an ES upgrade from v0.20.5 to v1.2.1 > I tested in a 2 node cluster, 3 indices, ~4 million docs, 18G file sizes, 20 > shards, 1 replicas > However, after bumping the version and reboot the cluster, I kept on seeing > some shards are damaged. The ES log said: > Caused by: org.apache.lucene.index.CorruptIndexException: did not read all > bytes from file: read 451 vs size 452 (resource: > BufferedChecksumIndexInput(MMapIndexInput(path="/18/index/\_195c\_i.del"))) > > This badly blocked the version upgrade in my case. > Could you any one point me the reason of this issue? > Lots appreciate to your help! > > Wei > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/e315ae9e-5c4a-43ab-a48f-3201dc52c6c9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e315ae9e-5c4a-43ab-a48f-3201dc52c6c9%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAMUKNZWZgg0RiaizYGaUyC2kWAXFAfSqM2knb7z5bkwNeth1HQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMUKNZWZgg0RiaizYGaUyC2kWAXFAfSqM2knb7z5bkwNeth1HQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:00am UTC](https://discuss.elastic.co/t/elasticsearch-version-upgrade-issue-corruptindexexception/19696/4 "2017-07-06T01:00:22Z")