Elastic not working cold nodes with S3 (minio)

Hello!
My problem is that I'm trying to put a date on S3 (mini). The directory is mounted locally. The path is specified in the config file. But when you try to run a node, it produces the following error.

> [2019-11-14T10:00:50,425][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [es-n1] uncaught exception in thread [main]

org.elasticsearch.bootstrap.StartupException: ElasticsearchException[java.io.IOException: failed to read /mnt/s3backup/cold-data/nodes/0/_state/global-0.st]; nested: IOException[failed to read /mnt/s3backup/cold-data/nodes/0/_state/global-0.st]; nested: CorruptStateException[org.apache.lucene.index.CorruptIndexException: codec footer mismatch (file truncated?): actual footer=-100926469 vs expected footer=-1071082520 (resource=BufferedChecksumIndexInput(SimpleFSIndexInput(path="/mnt/s3backup/cold-data/nodes/0/_state/global-0.st")))]; nested: CorruptIndexException[codec footer mismatch (file truncated?): actual footer=-100926469 vs expected footer=-1071082520 (resource=BufferedChecksumIndexInput(SimpleFSIndexInput(path="/mnt/s3backup/cold-data/nodes/0/_state/global-0.st")))];
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.4.1.jar:7.4.1]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.4.1.jar:7.4.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.4.1.jar:7.4.1]
Caused by: org.elasticsearch.ElasticsearchException: java.io.IOException: failed to read /mnt/s3backup/cold-data/nodes/0/_state/global-0.st
at org.elasticsearch.ExceptionsHelper.maybeThrowRuntimeAndSuppress(ExceptionsHelper.java:167) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.gateway.MetaDataStateFormat.loadGeneration(MetaDataStateFormat.java:414) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.gateway.MetaStateService.loadFullState(MetaStateService.java:81) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.gateway.GatewayMetaState.upgradeMetaData(GatewayMetaState.java:141) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.gateway.GatewayMetaState.(GatewayMetaState.java:95) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.node.Node.(Node.java:485) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.node.Node.(Node.java:255) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:221) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.4.1.jar:7.4.1]
... 6 more
Caused by: java.io.IOException: failed to read /mnt/s3backup/cold-data/nodes/0/_state/global-0.st
at org.elasticsearch.gateway.MetaDataStateFormat.loadGeneration(MetaDataStateFormat.java:408) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.gateway.MetaStateService.loadFullState(MetaStateService.java:81) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.gateway.GatewayMetaState.upgradeMetaData(GatewayMetaState.java:141) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.gateway.GatewayMetaState.(GatewayMetaState.java:95) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.node.Node.(Node.java:485) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.node.Node.(Node.java:255) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:221) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.4.1.jar:7.4.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.4.1.jar:7.4.1]
... 6 more
Caused by: org.elasticsearch.gateway.CorruptStateException: org.apache.lucene.index.CorruptIndexException: codec footer mismatch (file truncated?): actual footer=-100926469 vs expected footer=-1071082520 (resource=BufferedChecksumIndexInput(SimpleFSIndexInput(path="/mnt/s3backup/cold-data/nodes/0/_state/global-0.st")))
at org.elasticsearch.gateway.MetaDataStateFormat.read(MetaDataStateFormat.java:307) ~[elasticsearch-7.4.1.jar:7.4.1]

bash-4.2$ less /mnt/s3backup/cold-data/nodes/0/_state/global-0.st
"/mnt/s3backup/cold-data/nodes/0/_state/global-0.st" may be a binary file. See it anyway?

What can be done to correct this situation? Tell me, is it possible to add support for third-party S3 elasticsearch (for example minio)?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.