Docker elasticsearch cluster

I want to configure elasticsearch cluster in docker between 3 servers
node1, node2, node3 - docker-compose.yml

version: '2'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.0.2
    restart: always
    container_name: elasticsearch
    environment:
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms6g -Xmx6g"
      - xpack.security.enabled=false
      - http.host=0.0.0.0
      - transport.host=0.0.0.0
    ulimits:
      memlock:
        soft: -1
        hard: -1
    ports:
      - "9400:9200"
      - "9500:9300"
    volumes:
      - esdata:/usr/share/elasticsearch/data
      - ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - /backup/snapshot:/backup/snapshot

volumes:
  esdata:
    driver: local

config elasticsearch.yml
node1

cluster.name: docker
node.name: n1
network.host: 0.0.0.0
bootstrap.memory_lock: true
cluster.routing.allocation.disk.threshold_enabled: false
discovery.zen.ping.unicast.hosts: ["ip_node1:9500", "ip_node2:9500", "ip_node3:9500"]
discovery.zen.minimum_master_nodes: 2
network.publish_host: ip_node1

node2

cluster.name: docker
node.name: n2
network.host: 0.0.0.0
bootstrap.memory_lock: true
cluster.routing.allocation.disk.threshold_enabled: false
discovery.zen.ping.unicast.hosts: ["ip_node1:9500", "ip_node2:9500", "ip_node3:9500"]
discovery.zen.minimum_master_nodes: 2
network.publish_host: ip_node2

but getting errors

[2019-09-20T13:29:34,123][INFO ][o.e.x.m.e.Exporters      ] [n1] skipping exporter [default_local] as it isn't ready yet
[2019-09-20T13:29:34,124][ERROR][o.e.x.m.AgentService     ] [n1] exception when exporting documents
org.elasticsearch.xpack.monitoring.exporter.ExportException: exporters are either not ready or faulty
        at org.elasticsearch.xpack.monitoring.exporter.Exporters.export(Exporters.java:188) ~[x-pack-5.0.2.jar:5.0.2]
        at org.elasticsearch.xpack.monitoring.AgentService$ExportingWorker.run(AgentService.java:208) [x-pack-5.0.2.jar:5.0.2]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92-internal]

Hi @garcia

I would rather do a service for each one of the container. You can also configure more settings directly in the docker-compose.yml. Have you seen the following thread? Multi host elasticsearch cluster using docker

There is a link to further installation instructions on which my environment is based on.

Hope this helps.

MiTschMR

Hello @MiTschMR, thanks for the answer, but I already have 3 servers and on each server I want to raise elasticsearch in docker and raise a cluster between them

I have a problem now, there is docker-compose.yml the same for 3 servers

version: '2'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.0.2
    restart: always
    container_name: elasticsearch
    environment:
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms6g -Xmx6g"
      - xpack.security.enabled=false
    ulimits:
      memlock:
        soft: -1
        hard: -1
    ports:
      - "9400:9200"
      - "9500:9300"
    volumes:
      - esdata:/usr/share/elasticsearch/data
      - ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - /backup_es/snapshot:/backup_es/snapshot
volumes:
  esdata:
    driver: local

and a different elasticsearch.yml config for each server
elasticsearch.yml
node1

cluster.name: docker
node.name: n1
network.host: 0.0.0.0
bootstrap.memory_lock: true
cluster.routing.allocation.disk.threshold_enabled: false
discovery.zen.ping.unicast.hosts: ["ip_node1:9500", "ip_node2:9500", "ip_node3:9500"]
discovery.zen.minimum_master_nodes: 2
path.repo: ["/backup_es/snapshot"]

node2

cluster.name: docker
node.name: n2
network.host: 0.0.0.0
bootstrap.memory_lock: true
cluster.routing.allocation.disk.threshold_enabled: false
discovery.zen.ping.unicast.hosts: ["ip_node1:9500", "ip_node2:9500", "ip_node3:9500"]
discovery.zen.minimum_master_nodes: 2
path.repo: ["/backup_es/snapshot"]

node3

cluster.name: docker
node.name: n3
network.host: 0.0.0.0
bootstrap.memory_lock: true
cluster.routing.allocation.disk.threshold_enabled: false
discovery.zen.ping.unicast.hosts: ["ip_node1:9500", "ip_node2:9500", "ip_node3:9500"]
discovery.zen.minimum_master_nodes: 2
path.repo: ["/backup_es/snapshot"]

but when I start the first node, I get this error

[2019-09-22T10:51:36,815][INFO ][o.e.t.TransportService   ] [n1] publish_address {192.168.160.2:9300}, bound_addresses {0.0.0.0:9300}
[2019-09-22T10:51:36,818][INFO ][o.e.b.BootstrapCheck     ] [n1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2019-09-22T10:51:46,705][INFO ][o.e.x.m.e.Exporters      ] [n1] skipping exporter [default_local] as it isn't ready yet
[2019-09-22T10:51:46,706][ERROR][o.e.x.m.AgentService     ] [n1] exception when exporting documents
org.elasticsearch.xpack.monitoring.exporter.ExportException: exporters are either not ready or faulty
        at org.elasticsearch.xpack.monitoring.exporter.Exporters.export(Exporters.java:188) ~[x-pack-5.0.2.jar:5.0.2]
        at org.elasticsearch.xpack.monitoring.AgentService$ExportingWorker.run(AgentService.java:208) [x-pack-5.0.2.jar:5.0.2]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92-internal]

if I put discovery.zen.minimum_master_nodes: 1 then it starts without errors

if I add elasticsearch.yml in the config

xpack.monitoring.enabled: false

when starting the error above is no longer, but timeouts

[2019-09-22T10:58:59,799][INFO ][o.e.t.TransportService   ] [n1] publish_address {192.168.176.2:9300}, bound_addresses {0.0.0.0:9300}
[2019-09-22T10:58:59,803][INFO ][o.e.b.BootstrapCheck     ] [n1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2019-09-22T10:59:29,820][WARN ][o.e.n.Node               ] [n1] timed out while waiting for initial discovery state - timeout: 30s
[2019-09-22T10:59:29,834][INFO ][o.e.h.HttpServer         ] [n1] publish_address {192.168.176.2:9200}, bound_addresses {0.0.0.0:9200}
[2019-09-22T10:59:29,835][INFO ][o.e.n.Node               ] [n1] started
[2019-09-22T11:00:29,835][WARN ][r.suppressed             ] path: /_cat/nodes, params: {v=}
org.elasticsearch.discovery.MasterNotDiscoveredException
        at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:214) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.cluster.ClusterStateObserver$ContextPreservingListener.onTimeout(ClusterStateObserver.java:350) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:240) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.cluster.service.ClusterService$NotifyTimeout.run(ClusterService.java:957) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:458) [elasticsearch-5.0.2.jar:5.0.2]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_92-internal]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_92-internal]

Why are you running such an old version? Why not at least the most recent 5.x release?

the application is still running on version 5, is this a problem with the version?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.