S3 repository exception

Bubunia_Patra · April 23, 2019, 4:05pm

Hi all,

I am trying to setup elastic search (5.6.x) using S3 compatible storage such as minio as the backend.
I am following the steps mentioned at below link:

github.com

minio/cookbook/blob/master/docs/elasticsearch-snapshots-on-minio.md

# Elasticsearch snapshots on MinIO Server [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)

Elasticsearch is a distributed, RESTful search and analytics engine. In this recipe we'll see how to store Elasticsearch `6.x` snapshots to MinIO server.

## 1. Prerequisites

- Install MinIO Server from [here](https://docs.min.io/docs/minio-quickstart-guide).
- Download Elasticsearch version `6.2.4` from [here](https://www.elastic.co/downloads/elasticsearch).

## 2. Installation

Extract the `elasticsearch-6.2.4` tar ball in the directory where you plan to install Elasticsearch.

## 3. Install S3 Repository plugin in Elasticsearch

The S3 repository plugin adds support for using S3 as a repository for Snapshot/Restore. Install the plugin using plugin manager. Run the below command from inside the Elasticsearch directory

```sh
sudo bin/elasticsearch-plugin install repository-s3
```

This file has been truncated. show original

I am unable to create the repository (Step-6).

{"error":{"root_cause":[{"type":"repository_exception","reason":"[my_minio_repository] repository type [s3] does not exist"}],"type":"repository_exception","reason":"[my_minio_repository] repository type [s3] does not exist"},"status":500}

curl -X POST "localhost:9200/_snapshot/my_minio_repository" -H 'Content-Type: application/json' -d'
{ "type": "s3", "settings": {"bucket": "elasticsearch", "endpoint": "http://127.0.0.1:9000", "protocol": "http", "region": "us-east-1","access_key": "","secret_key": ""}}'

Could you please help in this regard?

Regards
Pradeep

Bubunia_Patra · April 24, 2019, 3:22pm

The following is the logs i get:

[2019-04-24T15:17:46,731][WARN ][o.e.r.RepositoriesService] [mma40wx] failed to create repository [my_minio_repository]
org.elasticsearch.repositories.RepositoryException: [my_minio_repository] repository type [s3] does not exist
at org.elasticsearch.repositories.RepositoriesService.createRepository(RepositoriesService.java:380) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.repositories.RepositoriesService.registerRepository(RepositoriesService.java:356) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.repositories.RepositoriesService.access$100(RepositoriesService.java:56) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.repositories.RepositoriesService$1.execute(RepositoriesService.java:109) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.service.ClusterService.executeTasks(ClusterService.java:634) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.service.ClusterService.calculateTaskOutputs(ClusterService.java:612) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.service.ClusterService.runTasks(ClusterService.java:571) [elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.service.ClusterService$ClusterServiceTaskBatcher.run(ClusterService.java:263) [elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:575) [elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:247) [elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:210) [elasticsearch-5.6.10.jar:5.6.10]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
[2019-04-24T15:17:46,739][WARN ][r.suppressed ] path: /_snapshot/my_minio_repository, params: {repository=my_minio_repository}
org.elasticsearch.repositories.RepositoryException: [my_minio_repository] repository type [s3] does not exist
at org.elasticsearch.repositories.RepositoriesService.createRepository(RepositoriesService.java:380) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.repositories.RepositoriesService.registerRepository(RepositoriesService.java:356) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.repositories.RepositoriesService.access$100(RepositoriesService.java:56) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.repositories.RepositoriesService$1.execute(RepositoriesService.java:109) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.service.ClusterService.executeTasks(ClusterService.java:634) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.service.ClusterService.calculateTaskOutputs(ClusterService.java:612) ~[elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.service.ClusterService.runTasks(ClusterService.java:571) [elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.service.ClusterService$ClusterServiceTaskBatcher.run(ClusterService.java:263) [elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:575) [elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:247) [elasticsearch-5.6.10.jar:5.6.10]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:210) [elasticsearch-5.6.10.jar:5.6.10]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]

DavidTurner · April 24, 2019, 3:34pm

This indicates you have not installed the repository-s3 plugin.

Bubunia_Patra · April 24, 2019, 3:59pm

I have installed the repository-s3 plugin already.

[elasticsearch@a3e18636c5fe ~] curl -X PUT http://192.168.0.100:9200/_snapshot/my_minio_repository -H 'Content-Type: application/json' -H 'Accept: application/json' -d '{ "type": "s3", "settings": { "bucket": "elasticsearch", "region": "us-east-1", "endpoint": "http://192.168.0.100:9000" } }' {"error":{"root_cause":[{"type":"repository_exception","reason":"[my_minio_repository] repository type [s3] does not exist"}],"type":"repository_exception","reason":"[my_minio_repository] repository type [s3] does not exist"},"status":500}[elasticsearch@a3e18636c5fe ~]
[elasticsearch@a3e18636c5fe ~] [elasticsearch@a3e18636c5fe ~]
[elasticsearch@a3e18636c5fe ~] [elasticsearch@a3e18636c5fe ~]
[elasticsearch@a3e18636c5fe ~] [elasticsearch@a3e18636c5fe ~] elasticsearch-plugin list
ingest-geoip
ingest-user-agent
repository-s3
x-pack
[elasticsearch@a3e18636c5fe ~]$

DavidTurner · April 24, 2019, 4:06pm

Can you try the following to see the plugins that are active on all the running nodes?

curl http://192.168.0.100:9200/_nodes?filter_path=nodes.*.plugins

dadoonet · April 24, 2019, 4:13pm

Probably you did not restart all nodes.

system · May 22, 2019, 4:13pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.