Docker-compose fails with ingest-attachment

I'm having real troubles trying to get the ingest-attachment plugin working with docker via docker-compose.

I can get ES working without any plugins using the following YML file

version: '3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.6.1
    container_name: elasticsearch
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata1:/usr/share/elasticsearch/test_es/data
    ports:
      - 9200:9200
  
volumes:
  esdata1:
    driver: local

This starts as expected and can be seen from the output of docker ps -a

CONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS                      PORTS                                                      NAMES
c4bbcfb01126        docker.elastic.co/elasticsearch/elasticsearch:6.6.1   "/usr/local/bin/dock…"   28 seconds ago      Up 24 seconds               0.0.0.0:9200->9200/tcp, 9300/tcp                           elasticsearch

But as soon as I add in the plugin elasticsearch exits with code 0. I'm adding the plugging by using

    command: bash -c "bin/elasticsearch-plugin install --batch ingest-attachment"

Here's the outupt of docker-compose -f example.yml up

elasticsearch    | -> Downloading ingest-attachment from elastic
elasticsearch    | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
elasticsearch    | @     WARNING: plugin requires additional permissions     @
elasticsearch    | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
elasticsearch    | * java.lang.RuntimePermission accessClassInPackage.sun.java2d.cmm.kcms
elasticsearch    | * java.lang.RuntimePermission accessDeclaredMembers
elasticsearch    | * java.lang.RuntimePermission getClassLoader
elasticsearch    | * java.lang.reflect.ReflectPermission suppressAccessChecks
elasticsearch    | * java.security.SecurityPermission createAccessControlContext
elasticsearch    | * java.security.SecurityPermission insertProvider
elasticsearch    | * java.security.SecurityPermission putProviderProperty.BC
elasticsearch    | See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
elasticsearch    | for descriptions of what these permissions allow and the associated risks.
elasticsearch    | -> Installed ingest-attachment
elasticsearch exited with code 0

So it seems to get through the install but then fails. Here's the full YML after the changes

version: '3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.6.1
    command: bash -c "bin/elasticsearch-plugin install --batch ingest-attachment"
    container_name: elasticsearch
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata1:/usr/share/elasticsearch/test_es/data
    ports:
      - 9200:9200
  
volumes:
  esdata1:
    driver: local

Has anyone managed to get it working? 'm new to using Docker so I suspect I am doing something wrong but this was taken from some SO examples of plugins

Update
I decide to change to 6.6.2 so that I would have a fresh image. Posting as a comment since it will be too long otherwise. I've also included the output from docker events as the logs show nothing

YML

version: '3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.6.2
    command: bash -c "bin/elasticsearch-plugin install --batch ingest-attachment"
    container_name: elasticsearch
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata1:/usr/share/elasticsearch/test_example2/data
    ports:
      - 9200:9200
    
volumes:
  esdata1:
    driver: local

** Running **

WARNING: The Docker Engine you're using is running in swarm mode.

Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.

To deploy your application across the swarm, use `docker stack deploy`.

Pulling elasticsearch (docker.elastic.co/elasticsearch/elasticsearch:6.6.2)...
6.6.2: Pulling from elasticsearch/elasticsearch
a02a4930cb5d: Already exists
1c0c2b94f1aa: Pull complete
e7ba1c987917: Pull complete
8a5898116619: Pull complete
fa49d069f225: Pull complete
0dee6ce7977c: Pull complete
77ecd20c6308: Pull complete
Recreating elasticsearch ... done
Attaching to elasticsearch
elasticsearch    | -> Downloading ingest-attachment from elastic
elasticsearch    | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
elasticsearch    | @     WARNING: plugin requires additional permissions     @
elasticsearch    | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
elasticsearch    | * java.lang.RuntimePermission accessClassInPackage.sun.java2d.cmm.kcms
elasticsearch    | * java.lang.RuntimePermission accessDeclaredMembers
elasticsearch    | * java.lang.RuntimePermission getClassLoader
elasticsearch    | * java.lang.reflect.ReflectPermission suppressAccessChecks
elasticsearch    | * java.security.SecurityPermission createAccessControlContext
elasticsearch    | * java.security.SecurityPermission insertProvider
elasticsearch    | * java.security.SecurityPermission putProviderProperty.BC
elasticsearch    | See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
elasticsearch    | for descriptions of what these permissions allow and the associated risks.
elasticsearch    | -> Installed ingest-attachment
elasticsearch exited with code 0

Third part as the post is too long

** Docker Events **

2019-03-23T12:33:16.587939179Z image pull docker.elastic.co/elasticsearch/elasticsearch:6.6.2 (license=Elastic License, name=docker.elastic.co/elasticsearch/elasticsearch, org.label-schema.build-date=20181205, org.label-schema.license=GPLv2, org.label-schema.name=elasticsearch, org.label-schema.schema-version=1.0, org.label-schema.url=https://www.elastic.co/products/elasticsearch, org.label-schema.vcs-url=https://github.com/elastic/elasticsearch, org.label-schema.vendor=Elastic, org.label-schema.version=6.6.2)
2019-03-23T12:33:16.624879935Z container rename 67acd0464f92a6987d57015ec8f5b723510b584363211addb12dce5718b97e54 (com.docker.compose.config-hash=3a13bb3916e52d3d660d730c0e49ac9e09353634181308680a1ee751241859ae, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=documents, com.docker.compose.service=elasticsearch, com.docker.compose.version=1.23.2, image=docker.elastic.co/elasticsearch/elasticsearch:6.6.1, license=Elastic License, name=67acd0464f92_elasticsearch, oldName=/elasticsearch, org.label-schema.build-date=20181205, org.label-schema.license=GPLv2, org.label-schema.name=elasticsearch, org.label-schema.schema-version=1.0, org.label-schema.url=https://www.elastic.co/products/elasticsearch, org.label-schema.vcs-url=https://github.com/elastic/elasticsearch, org.label-schema.vendor=Elastic, org.label-schema.version=6.6.1)
2019-03-23T12:33:17.173767374Z volume create documents_esdata1 (driver=local)
2019-03-23T12:33:17.245901821Z container create b99c56a2312fdb0d6ec98821b41a33e9bcfab8adc16c8ad24f0c333b847efd7d (com.docker.compose.config-hash=30745264fcb36f1a0046240528370ff19d565b422a7c022e5eee4dc458eceba4, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=documents, com.docker.compose.service=elasticsearch, com.docker.compose.version=1.23.2, image=docker.elastic.co/elasticsearch/elasticsearch:6.6.2, license=Elastic License, name=elasticsearch, org.label-schema.build-date=20181205, org.label-schema.license=GPLv2, org.label-schema.name=elasticsearch, org.label-schema.schema-version=1.0, org.label-schema.url=https://www.elastic.co/products/elasticsearch, org.label-schema.vcs-url=https://github.com/elastic/elasticsearch, org.label-schema.vendor=Elastic, org.label-schema.version=6.6.2)
2019-03-23T12:33:17.248774307Z container attach b99c56a2312fdb0d6ec98821b41a33e9bcfab8adc16c8ad24f0c333b847efd7d (com.docker.compose.config-hash=30745264fcb36f1a0046240528370ff19d565b422a7c022e5eee4dc458eceba4, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=documents, com.docker.compose.service=elasticsearch, com.docker.compose.version=1.23.2, image=docker.elastic.co/elasticsearch/elasticsearch:6.6.2, license=Elastic License, name=elasticsearch, org.label-schema.build-date=20181205, org.label-schema.license=GPLv2, org.label-schema.name=elasticsearch, org.label-schema.schema-version=1.0, org.label-schema.url=https://www.elastic.co/products/elasticsearch, org.label-schema.vcs-url=https://github.com/elastic/elasticsearch, org.label-schema.vendor=Elastic, org.label-schema.version=6.6.2)
2019-03-23T12:33:17.256514654Z network disconnect 71d82f5b37e0c5ea822a34e43fa201af6175da42fd09c455cf21f9b08e2f70eb (container=b99c56a2312fdb0d6ec98821b41a33e9bcfab8adc16c8ad24f0c333b847efd7d, name=documents_default, type=bridge)
2019-03-23T12:33:17.293477232Z network connect 71d82f5b37e0c5ea822a34e43fa201af6175da42fd09c455cf21f9b08e2f70eb (container=b99c56a2312fdb0d6ec98821b41a33e9bcfab8adc16c8ad24f0c333b847efd7d, name=documents_default, type=bridge)
2019-03-23T12:33:17.296843556Z volume mount documents_esdata1 (container=b99c56a2312fdb0d6ec98821b41a33e9bcfab8adc16c8ad24f0c333b847efd7d, destination=/usr/share/elasticsearch/unittest_nursdoc/data, driver=local, propagation=, read/write=true)
2019-03-23T12:33:18.098931609Z container start b99c56a2312fdb0d6ec98821b41a33e9bcfab8adc16c8ad24f0c333b847efd7d (com.docker.compose.config-hash=30745264fcb36f1a0046240528370ff19d565b422a7c022e5eee4dc458eceba4, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=documents, com.docker.compose.service=elasticsearch, com.docker.compose.version=1.23.2, image=docker.elastic.co/elasticsearch/elasticsearch:6.6.2, license=Elastic License, name=elasticsearch, org.label-schema.build-date=20181205, org.label-schema.license=GPLv2, org.label-schema.name=elasticsearch, org.label-schema.schema-version=1.0, org.label-schema.url=https://www.elastic.co/products/elasticsearch, org.label-schema.vcs-url=https://github.com/elastic/elasticsearch, org.label-schema.vendor=Elastic, org.label-schema.version=6.6.2)
2019-03-23T12:33:18.109045406Z container destroy 67acd0464f92a6987d57015ec8f5b723510b584363211addb12dce5718b97e54 (com.docker.compose.config-hash=3a13bb3916e52d3d660d730c0e49ac9e09353634181308680a1ee751241859ae, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=documents, com.docker.compose.service=elasticsearch, com.docker.compose.version=1.23.2, image=docker.elastic.co/elasticsearch/elasticsearch:6.6.1, license=Elastic License, name=67acd0464f92_elasticsearch, org.label-schema.build-date=20181205, org.label-schema.license=GPLv2, org.label-schema.name=elasticsearch, org.label-schema.schema-version=1.0, org.label-schema.url=https://www.elastic.co/products/elasticsearch, org.label-schema.vcs-url=https://github.com/elastic/elasticsearch, org.label-schema.vendor=Elastic, org.label-schema.version=6.6.1)
2019-03-23T12:33:24.710977580Z container die b99c56a2312fdb0d6ec98821b41a33e9bcfab8adc16c8ad24f0c333b847efd7d (com.docker.compose.config-hash=30745264fcb36f1a0046240528370ff19d565b422a7c022e5eee4dc458eceba4, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=documents, com.docker.compose.service=elasticsearch, com.docker.compose.version=1.23.2, exitCode=0, image=docker.elastic.co/elasticsearch/elasticsearch:6.6.2, license=Elastic License, name=elasticsearch, org.label-schema.build-date=20181205, org.label-schema.license=GPLv2, org.label-schema.name=elasticsearch, org.label-schema.schema-version=1.0, org.label-schema.url=https://www.elastic.co/products/elasticsearch, org.label-schema.vcs-url=https://github.com/elastic/elasticsearch, org.label-schema.vendor=Elastic, org.label-schema.version=6.6.2)
2019-03-23T12:33:24.993745885Z network disconnect 71d82f5b37e0c5ea822a34e43fa201af6175da42fd09c455cf21f9b08e2f70eb (container=b99c56a2312fdb0d6ec98821b41a33e9bcfab8adc16c8ad24f0c333b847efd7d, name=documents_default, type=bridge)
2019-03-23T12:33:25.092380891Z volume unmount documents_esdata1 (container=b99c56a2312fdb0d6ec98821b41a33e9bcfab8adc16c8ad24f0c333b847efd7d, driver=local)

For anyone reading this I solved it by creating a custom image using Dockerfile

Dockerfile Content

FROM elasticsearch:6.6.2
RUN bin/elasticsearch-plugin install --batch ingest-attachment

Then create your local image and tag using docker build -t <tagname> .

Then edited the YML file and changed the image to image: elasticsearchingest:latest

2 Likes

Thanks for sharing your solution.

For tests, here is what I'm doing. It's more manual than your solution though:

---
version: '3'
services:

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:$ELASTIC_VERSION
    environment:
      - bootstrap.memory_lock=true
      - discovery.type=single-node
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata:/usr/share/elasticsearch/data
      - esplugins:/usr/share/elasticsearch/plugins
    ports:
      - 9200:9200

volumes:
  esdata:
  esplugins:

Then

docker exec -i -t 6x_elasticsearch_1 /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-attachment

And then I restart docker-compose.

Very manual as I said :slight_smile:

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.