Elastic and LocalStack

[Sergiodive]

Hello
I am running Elasticsearch 7.10 with a localstack s3 instance. The issue is when i try to register a repository. If i do this

  curl -X PUT "localhost:9200/_snapshot/repo1?pretty" -H 'Content-Type: application/json' -d'
  {
      "type": "s3",
      "settings": {
        "bucket": "elasticbucket",
        "endpoint": "http://localhost:4566"
     }
}'

I returns

type" : "repository_verification_exception",
"reason" : "[repo1] path  is not accessible on master node"

And the bucket is there.

aws --endpoint-url=http://localhost:4566  s3 ls                                            
2021-02-09 14:57:31 elasticbucket

The only thing i set in the elastic container is this

bin/elasticsearch-keystore add s3.client.default.access_key
bin/elasticsearch-keystore add s3.client.default.secret_key

I also set the network mode to host for both containers

I had this issue last week but after I shut my machine down and tried again it magically worked. I don't know why. Today I had to recreate the local stack and elastic containers and I am getting this issue again.

I am quite new to all of this so I am not familiar with the tools to trouble shoot. I read some posts on here related to this and I believe this is NOT related to IAM because I never had set that when it was working

Last thing. The s3 plugin is included the Dockerfile I created.
This is the dockerfile

FROM docker.elastic.co/elasticsearch/elasticsearch:7.10.2
COPY --chown=elasticsearch:elasticsearch elasticsearch.yml /usr/share/elasticsearch/config/
RUN bin/elasticsearch-plugin install --batch repository-s3

When I build it I get this warning

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,resolve
* java.util.PropertyPermission es.allow_insecure_settings read,write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

This is what my elasticseach.yml file looks like

cluster.name: "docker-cluster"
network.host: 0.0.0.0
path.repo: ["/usr/share/elasticsearch/data"]
s3.client.default.endpoint: localhost:4566
s3.client.default.protocol: http

and this is my run command

docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --mount type=bind,source=$(pwd),target=/usr/share/elasticsearch/data elastic01

I can list the bucket while inside the elastic container

[root@ubu0850 bin]# aws --endpoint-url=http://localhost:4566  s3 ls
2021-02-09 21:12:04 elasticbucket
[root@ubu0850 bin]# exit

Any Help is greatly appreciated

[Sergiodive]

I figured it out.
I had two problems
Problem one was for network mode I had to set it to host instead of bridge. The second problem was the s3 bucket has naming requirements
https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-s3-bucket-naming-requirements.html

Here is my docker compose should anyone ever need it

version: '3'
services:
  elasticsearch:
    build: .
    container_name: elasticsearch
    network_mode: host
    volumes:
      - ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./elasticsearch.keystore:/usr/share/elasticsearch/config/elasticsearch.keystore
      -  ./:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    environment:
      discovery.type: single-node
   # links:
    #  - "localstack:localstack"
    restart: always

  localstack:
    image: localstack/localstack:0.12.6
    container_name: localstack
    network_mode: host
    ports:
      - "4566:4566"
      - "4571:4571"
      - "8055-8080:8055-8080"
    environment:
      - SERVICES=s3
      - DEFAULT_REGION=us-east-1
      - DATA_DIR=/tmp/localstack/data
      - AWS_ACCESS_KEY_ID=1234
      - AWS_SECRET_ACCESS_KEY=1234
    volumes:
      - './.localstack:/tmp/localstack'
      - "./scripts/:/docker-entrypoint-initaws.d"