ElasticSearch Dockercompose with plugin not starting up

[gjahagir]

Hi,

I followed the article here to start a multi node cluster using docker compose.
i also installed mapper-size plugin for each node, manually using command -

docker exec es02 elasticsearch-plugin install mapper-size

I then enabled _size in kibana (metadata) and created an index with _size mapping, indexed few documents for testing around the doc size etc.

The issue now is when i do docker-compose down and then docker-compose up -d. The containers wont start and fail with error -

"ERROR.message":"Failed to parse mappings for index [[testindex/1I5u-GDvTmaUGWru35Dclg]]"

I think it has something to do around _size mapping and mapper size plug-in. Any help on this please...?

[jacinto_calvo_sintes]

Hi,
you'll have to install the plugins every time after you do a docker-compose down as they're not saved in the docker image.
You can either save the docker image once you've installed the plugins so you don't need to do it every time, or modify the docker-compose by adding the install plugin on each Elasticsearch node similar to this.

 elasticsearch:
     environment:
            - "discovery.type=single-node"
            - "MAX_CLAUSE_COUNT=4096"
            - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
     image: docker.elastic.co/elasticsearch/elasticsearch:7.16.1
     command:
      - sh
      - -c
      - "./bin/elasticsearch-plugin install plugin1; ./bin/elasticsearch-plugin install plugin2; /usr/local/bin/docker-entrypoint.sh elasticsearch -E \"discovery.type=single-node\""
     ports:
         - 9200:9200

[gjahagir]

Hi @jacinto_calvo_sintes thanks for your reply.
I tried your approach by adding the command node for each elasticsearch node in yml file.

es03:
    depends_on:
      - es02
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata03:/usr/share/elasticsearch/data
    command:
      - sh
      - -c
      - "./bin/elasticsearch-plugin install mapper-size"
    environment:
      - node.name=es03
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es01,es02
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es03/es03.key
      - xpack.security.http.ssl.certificate=certs/es03/es03.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.http.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es03/es03.key
      - xpack.security.transport.ssl.certificate=certs/es03/es03.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

however, i noticed that the containers exit immediately after they are created. The console in docker desktop shows

-> Installing mapper-size

-> Downloading mapper-size from elastic

[=================================================] 100%??

-> Installed mapper-size

-> Please restart Elasticsearch to activate any plugins installed

i noticed that it requires restart of the container. how do i do that in the yml file?

[jacinto_calvo_sintes]

Hi @gjahagir,
You need to later run the docker-entrypoint as with the install command you're overwriting the default action from the container when it starts.

In mi case, I run it after the plugin install command. I think some of the environment variables are lost when running from the command and had to add them with the -E option.

- "./bin/elasticsearch-plugin install plugin1; /usr/local/bin/docker-entrypoint.sh elasticsearch -E \"discovery.type=single-node\""

Hope this helps.