We are trying to in-place upgradation of our elasticsearch cluster. We operate on timesharded index so what we were thinking is currently
- the cluster is on ES-7.17.6 version.
- We will upgrade the cluster to ES-8.11.1 and check if it is backward compatible.
- Now the whenever the new time sharded index is formed it will form with synthetic source since we are 8.11.1 cluster and we have defined such mapping.
But in the second step upon updating the cluster from 7.17.6 to 8.11.1 we are getting following error:
org.elasticsearch.ElasticsearchException: failed to bind service
at org.elasticsearch.node.Node.<init>(Node.java:1231) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.node.Node.<init>(Node.java:345) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:236) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:236) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:73) ~[elasticsearch-8.11.1-3.jar:?]
Caused by: java.io.IOException: Invalid vInt detected (too many bits)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:139) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.store.DataInput.readString(DataInput.java:289) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.store.DataInput.readMapOfStrings(DataInput.java:324) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.backward_codecs.lucene60.Lucene60FieldInfosFormat.readFieldInfos(Lucene60FieldInfosFormat.java:184) ~[lucene-backward-codecs-9.8.0-6.jar:?]
at org.apache.lucene.backward_codecs.lucene60.Lucene60FieldInfosFormat.read(Lucene60FieldInfosFormat.java:148) ~[lucene-backward-codecs-9.8.0-6.jar:?]
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:108) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:94) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:94) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:77) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:826) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:67) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:60) ~[lucene-core-9.8.0-6.jar:?]
at org.elasticsearch.gateway.PersistedClusterStateService.nodeMetadata(PersistedClusterStateService.java:351) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.env.NodeEnvironment.checkForIndexCompatibility(NodeEnvironment.java:510) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.env.NodeEnvironment.upgradeLegacyNodeFolders(NodeEnvironment.java:417) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:310) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.node.Node.<init>(Node.java:500) ~[elasticsearch-8.11.1-3.jar:?]
... 4 more
Suppressed: org.apache.lucene.index.CorruptIndexException: checksum passed (2ca3b2c3). possibly transient resource issue, or a Lucene or JVM bug (resource=Endianness reverser Checksum Index Input wrapper)
at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:500) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.backward_codecs.lucene60.Lucene60FieldInfosFormat.read(Lucene60FieldInfosFormat.java:153) ~[lucene-backward-codecs-9.8.0-6.jar:?]
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:108) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:94) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:94) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:77) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:826) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:67) ~[lucene-core-9.8.0-6.jar:?]
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:60) ~[lucene-core-9.8.0-6.jar:?]
at org.elasticsearch.gateway.PersistedClusterStateService.nodeMetadata(PersistedClusterStateService.java:351) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.env.NodeEnvironment.checkForIndexCompatibility(NodeEnvironment.java:510) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.env.NodeEnvironment.upgradeLegacyNodeFolders(NodeEnvironment.java:417) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:310) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.node.Node.<init>(Node.java:500) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.node.Node.<init>(Node.java:345) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:236) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:236) ~[elasticsearch-8.11.1-3.jar:?]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:73) ~[elasticsearch-8.11.1-3.jar:?]
Unable to figure out why? Since we have tried restoring indices from ES7 cluster to ES8 cluster and that works perfectly fine. Does this mean ES8 is not backward compatible with ES7? Or is there a way we can fix this?