Can't get the Enterprise Search service up with Docker

justaniceguy · February 10, 2021, 6:18pm

I am trying to deploy Elastic Enterprise but when I add the secret key... I mean, I think this needs to be an array because this is what it says here, but I a get this error message:

root@localhost:~# docker-compose -f elasticsearch-docker-compose.yml up -d

ERROR: The Compose file './elasticsearch-docker-compose.yml' is invalid because:
services.enterprisesearch.environment contains {"secret_management.encryption_keys": ["4096ea199bb51beb77d815c5da18d1aebb6e4a167dcc9b1ebd8b27723950d294"]}, which is an invalid type, it should be a string

And if I check the logs, it says:

root@localhost:~# docker logs root_enterprisesearch_1

Found java executable in PATH
Java version detected: 1.8.0_252 (major version: 8)
Enterprise Search is starting...

Invalid config file (/usr/share/enterprise-search/config/enterprise-search.yml):
The setting '#/secret_management/encryption_keys' is not valid
No secret management encryption keys were provided.
Your secrets cannot be stored unencrypted.
You can use the following generated encryption key in your config file to store new encrypted secrets:

secret_management.encryption_keys: [357287e82d94dc31af06bea068ed65b297d5bc8f0a692cc48858cc9636f34950]

It doesn't matter if I use the encription key suggested, I get the same error message.

EDIT:

docker-compose.yml

version: "3.7"

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.2
    container_name: elasticsearch
    restart: always
    environment:
      - xpack.security.enabled=false
      - discovery.type=single-node
      - node.name=es-node-1
      - cluster.name=app-search-docker-cluster
      - bootstrap.memory_lock=true
#      - elasticsearch.username:elastic
#      - elasticsearch.password:changeme
      - ES_JAVA_OPTS=-Xms512m -Xmx512m
      - elasticsearch.host:http://xxx.xxx.xxx.xxx:9200 # My IP, replaced for security
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    cap_add:
      - IPC_LOCK
    volumes:
      - elasticsearch-data-volume:/usr/share/elasticsearch/data
    ports:
      - 9200:9200

  enterprisesearch:
    image: docker.elastic.co/enterprise-search/enterprise-search:7.10.2
    environment:
      - elasticsearch.host=http://elasticsearch:9200
      - allow_es_settings_modification=true
      - JAVA_OPTS=-Xms2g -Xmx2g
      - ent_search.auth.source:standard
      - ent_search.external_url:http://xxx.xxx.xxx.xxx:3002 # My IP, replaced for security
      - ent_search.listen_host:xxx.xxx.xxx.xxx # My IP, replaced for security
      - ent_search.listen_port:3002
      - secret_management.encryption_keys:[4096ea199bb51beb77d815c5da18d1aebb6e4a167dcc9b1ebd8b27723950d294]
    ports:
      - 3002:3002
    depends_on:
      - elasticsearch

  kibana:
    container_name: kibana
    image: docker.elastic.co/kibana/kibana:7.10.2
    restart: always
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
      - enterpriseSearch.host:http://xxx.xxx.xxx.xxx:3002 # My IP, replaced for security
    ports:
      - 5601:5601
    depends_on:
      - elasticsearch  
      - enterprisesearch


volumes:
  elasticsearch-data-volume:
    driver: local

stephenb · February 11, 2021, 1:59am

Hi @justaniceguy

I am a little unclear where you are at in the process....

To Run Elasticsearch Enterprise Search you already need and Elasticsearch cluster up and running or are you trying do do this all in one docker compose file?

Perhaps If you share your docker compose we can help.

I just ran a docker compose with Kibana and ES and all running local and then just ran this seperate docker command to just check and it all came up (takes a little while) as shown here

docker run -p 3002:3002 \
-e elasticsearch.host='http://host.docker.internal:9200' \
-e elasticsearch.username=elastic \
-e elasticsearch.password=changeme \
-e allow_es_settings_modification=true \
-e secret_management.encryption_keys='[4a2cd3f81d39bf28738c10db0ca782095ffac07279561809eecc722e0c20eb09]' \
docker.elastic.co/enterprise-search/enterprise-search:7.11.0

justaniceguy · February 11, 2021, 2:46am

Thanks for your help @stephenb

I am trying to use just one docker compose file. Edited my first post to include the .yml file there.

stephenb · February 11, 2021, 4:12am

So here is my basic basic compose... it works... takes a while to come up.
You will then need start to layer on security assuming you want to.

---
version: '3'
services:

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.2
    container_name: es01
    environment:
      - bootstrap.memory_lock=true
      - discovery.type=single-node
      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
      - cluster.routing.allocation.disk.threshold_enabled=false
      - xpack.license.self_generated.type=trial
    ulimits:
      memlock:
        soft: -1
        hard: -1
    networks: ['stack']
    ports: ['9200:9200']

  enterprisesearch:
    image: docker.elastic.co/enterprise-search/enterprise-search:7.10.2
    container_name: ent01
    environment:
    - "elasticsearch.username=elastic"
    - "elasticsearch.password=changeme"
    - "elasticsearch.host=http://elasticsearch:9200"
    - "allow_es_settings_modification=true"
    - "secret_management.encryption_keys=[salkdfjhasldfkjhasdflkasjdhflaskdjfhasldkfjshad]"
    ports: ['3002:3002']
    networks: ['stack']
    links: ['elasticsearch']
    depends_on: ['elasticsearch']

  kibana:
    image: docker.elastic.co/kibana/kibana:7.10.2
    container_name: kib01
    ports: ['5601:5601']
    networks: ['stack']
    depends_on: ['elasticsearch', 'enterprisesearch']

networks:
  stack: {}

justaniceguy · February 11, 2021, 2:40pm

Thank you very much @stephenb

The docker-compose.yml file you created didn't work as expected, so I made a combination of yours and mine.

The result? Now Enterprise Search looks to be up but for some reason when I hit the endpoint its endpoint (http://xx.xx.xxx.xxx:3002) it redirects me to http://localhost:3002/welcome

When I hit the endpoint, this its creae thie log

ent_search | [2021-02-11T13:55:39.150+00:00][1][2500][connectors][INFO]: [Worker 0c86a44b96c1] The system is in read-only mode, cannot claim job; retrying in 60 seconds
ent_search | [2021-02-11T13:55:52.013+00:00][1][2504][app-server][INFO]: [1d214313-ddd2-472c-9aa2-56b65bfbb8ca] Started GET "/" for 186.6.254.48 at 2021-02-11 13:55:52 +0000
ent_search | [2021-02-11T13:55:52.109+00:00][1][2504][action_controller][INFO]: [1d214313-ddd2-472c-9aa2-56b65bfbb8ca] Processing by SharedTogo::HomeController#index as HTML
ent_search | [2021-02-11T13:55:52.111+00:00][1][2504][action_controller][INFO]: [1d214313-ddd2-472c-9aa2-56b65bfbb8ca] Parameters: {"host"=>"localhost:3002", "protocol"=>"http"}
ent_search | [2021-02-11T13:55:52.173+00:00][1][2504][action_controller][INFO]: [1d214313-ddd2-472c-9aa2-56b65bfbb8ca] Redirected to http://localhost:3002/welcome
ent_search | [2021-02-11T13:55:52.233+00:00][1][2504][action_controller][INFO]: [1d214313-ddd2-472c-9aa2-56b65bfbb8ca] Completed 302 Found in 121ms

This is my current docker-compose.yml file:

version: "3.7"

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.2
    container_name: elasticsearch
    restart: always
    environment:
      - xpack.security.enabled=false
      - discovery.type=single-node
      - node.name=es-node-1
      - cluster.name=app-search-docker-cluster
      - bootstrap.memory_lock=true
#      - elasticsearch.username:elastic
#      - elasticsearch.password:changeme
      - ES_JAVA_OPTS=-Xms512m -Xmx512m
      - elasticsearch.host:http://xx.xx.xxx.xxx:9200
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    cap_add:
      - IPC_LOCK
    volumes:
      - elasticsearch-data-volume:/usr/share/elasticsearch/data
    ports:
      - 9200:9200

  enterprisesearch:
    image: docker.elastic.co/enterprise-search/enterprise-search:7.10.2
    environment:
      - elasticsearch.host=http://elasticsearch:9200
      - allow_es_settings_modification=true
      - JAVA_OPTS=-Xms2g -Xmx2g
      - ent_search.auth.source:standard
      - ent_search.external_url:http://xx.xx.xxx.xxx:3002
      - ent_search.listen_host:xx.xx.xxx.xxx
      - ent_search.listen_port:3002
      - secret_management.encryption_keys:'[4096ea199bb51beb77d815c5da18d1aebb6e4a167dcc9b1ebd8b27723950d294]'
    ports:
      - 3002:3002
    depends_on:
      - elasticsearch

  kibana:
    container_name: kibana
    image: docker.elastic.co/kibana/kibana:7.10.2
    restart: always
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200    # address of elasticsearch docker container which kibana will connect
      - enterpriseSearch.host:http://xx.xx.xxx.xxx:3002
    ports:
      - 5601:5601
    depends_on:
      - elasticsearch                                   # kibana will start when elasticsearch has started
      - enterprisesearch


volumes:
  elasticsearch-data-volume:
    driver: local

EDIT:

Solved, by changing:

      - ent_search.external_url:http://xx.xx.xxx.xxx:3002
      - ent_search.listen_host:xx.xx.xxx.xxx
      - ent_search.listen_port:3002

For

    - "ent_search.external_url=http://xx.xx.xxx.xxx:3002"

Topic		Replies	Views
Running ELK stack with Enterprise Search using Docker on a server (Linux) invalid encryptions key Elastic Search docker , elastic-app-search	2	472	August 19, 2022
Enterprise Search - Docker Compose Elastic Search docker	3	1801	November 4, 2022
Unable to connect enterprise search running through docker to elastic search instance running in AWS Elastic Search	8	534	March 8, 2021
Enterprise Search - Error: key not found: "plugins" at org/jruby/RubyHash.java:1261:in `fetch' Elastic Search docker	2	297	November 4, 2022
Enterprise-search.yml is not a valid YAML file Elastic Search elastic-workplace-search	3	2022	June 22, 2020

Can't get the Enterprise Search service up with Docker

Related topics