I'm using the elasticsearch-py client to shrink old indices.
basically, I was executing this command:
settingsShrink = {
"settings": {
"index.number_of_replicas": 1,
"index.number_of_shards": 1,
"index.codec": "best_compression",
"index.routing.allocation.require._name": null,
"index.blocks.write": null
}
}
es.indices.shrink(index=elasticFinalIndex, target=elasticFinalShrinkIndex, body=settingsShrink, wait_for_active_shards=1)
I read that maybe using forcemerge command I could fix this issue, but when I try it I cannot decrease segments to just 1 using the command:
es.indices.forcemerge(index=elasticFinalIndex,max_num_segments=1)
This should be the full stack trace:
[2018-10-05T09:50:05,019][WARN ][o.e.i.c.IndicesClusterStateService] [saelk1] [[sroger-gtw-xfb-2018.02.12][0]] marking and sending shard failed due to [failed recovery]
org.elasticsearch.indices.recovery.RecoveryFailedException: [sroger-gtw-xfb-2018.02.12][0]: Recovery failed on {saelk1}{jSe6gdpyTuqBxfScv7SzIQ}{5rJk8R3gSOa_09IkBsZzug}{10.0.18.151}{10.0.18.151:9300}{ml.machine_memory=16828035072, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$8(IndexShard.java:2090) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:626) [elasticsearch-6.3.2.jar:6.3.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_65]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_65]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_65]
Caused by: org.elasticsearch.index.shard.IndexShardRecoveryException: failed recovery
at org.elasticsearch.index.shard.StoreRecovery.executeRecovery(StoreRecovery.java:343) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.shard.StoreRecovery.recoverFromLocalShards(StoreRecovery.java:123) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.shard.IndexShard.recoverFromLocalShards(IndexShard.java:1563) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$8(IndexShard.java:2085) ~[elasticsearch-6.3.2.jar:6.3.2]
... 4 more
Caused by: java.lang.IllegalArgumentException: Cannot use addIndexes(Directory) with indexes that have been created by a different Lucene version. The current index was generated by Lucene 6 while one of the directories contains an index that was generated with Lucene 7
at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:2850) ~[lucene-core-7.3.1.jar:7.3.1 ae0705edb59eaa567fe13ed3a222fdadc7153680 - caomanhdat - 2018-05-09 09:27:24]
at org.elasticsearch.index.shard.StoreRecovery.addIndices(StoreRecovery.java:170) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.shard.StoreRecovery.lambda$recoverFromLocalShards$3(StoreRecovery.java:131) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.shard.StoreRecovery.executeRecovery(StoreRecovery.java:301) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.shard.StoreRecovery.recoverFromLocalShards(StoreRecovery.java:123) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.shard.IndexShard.recoverFromLocalShards(IndexShard.java:1563) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$8(IndexShard.java:2085) ~[elasticsearch-6.3.2.jar:6.3.2]
... 4 more