Unable to create elasticsearch cluster in docker swarm

Elastic Stack Elasticsearch
1

i am trying to create a 2 node docker swarm elasticsearch cluster. have been trying this from 2 days but couldn't succeed. please help me. below is the compose file, first node joins the cluster but second not able to join. When i run the 2 nodes in same cluster 2 nodes becoming cluster but getting below error on second node when i run them in 2 separate host docker nodes.

bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [1] bootstrap checks failed.

Below is the compose file, any suggestions would be great help!
""""
version: '3.7'
services:
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.1
container_name: es01
environment:
- node.name=es01
- cluster.name=es-docker-cluster
- bootstrap.memory_lock=false
- network.host=0.0.0.0
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- discovery.zen.ping.unicast.hosts=es01,es02
- discovery.zen.minimum_master_nodes=1
- node.master=true
- node.data=true
- node.ingest=true
volumes:
- /opt/es01:/usr/share/elasticsearch/data
ports:
- 9200:9200
deploy:
placement:
constraints:
- "node.hostname == node1"
es02:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.1
container_name: es02
environment:
- node.name=es02
- cluster.name=es-docker-cluster
- bootstrap.memory_lock=false
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- discovery.zen.ping.unicast.hosts=es01,es02
- discovery.zen.minimum_master_nodes=1
- node.master=false
- node.data=true
- node.ingest=true
volumes:
- /opt/es02:/usr/share/elasticsearch/data
deploy:
placement:
constraints:
- "node.hostname == node2"
""""

2

What is the rest of the logs?

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

This is the icon to use if you are not using markdown format:

image
image764×92 16.4 KB

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
Please update your post.

4

Please don't post images of text as they are hard to read, may not display correctly for everyone, and are not searchable.

Instead, paste the text and format it with </> icon or pairs of triple backticks (```), and check the preview window to make sure it's properly formatted before posting it. This makes it more likely that your question will receive a useful answer.

It would be great if you could update your post to solve this.

6

Below is the docker compose file which i am using deploy it as swarm.

version: '3.7'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.6.1
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - bootstrap.memory_lock=false
      - network.host=0.0.0.0
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - discovery.zen.ping.unicast.hosts=es01,es02
      - discovery.zen.minimum_master_nodes=1
      - node.master=true
      - node.data=true
      - node.ingest=true
    volumes:
      - /opt/es01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    deploy:
      placement:
        constraints:
          - "node.hostname == node1"
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.6.1
    container_name: es02
    environment:
      - node.name=es02
      - cluster.name=es-docker-cluster
      - bootstrap.memory_lock=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - discovery.zen.ping.unicast.hosts=es01,es02
      - discovery.zen.minimum_master_nodes=1
      - node.master=false
      - node.data=true
      - node.ingest=true
    volumes:
      - /opt/es02:/usr/share/elasticsearch/data
    deploy:
      placement:
        constraints:
          - "node.hostname == node2"

Here are the logs from node1:

{"type": "server", "timestamp": "2020-03-30T06:30:40,328Z", "level": "INFO", "component": "o.e.c.s.MasterService", "cluster.name": "es-docker-cluster", "node.name": "es01", "message": "elected-as-master ([1] nodes joined)[{es01}{TxAWx5BJRKWaFA4_n7dIDA}{OlRlsCl9SrCT9qNDlbIjSQ}{10.0.61.3}{10.0.61.3:9300}{dilm}{ml.machine_memory=135024914432, xpack.installed=true, ml.max_open_jobs=20} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 9, version: 40, delta: master node changed {previous [], current [{es01}{TxAWx5BJRKWaFA4_n7dIDA}{OlRlsCl9SrCT9qNDlbIjSQ}{10.0.61.3}{10.0.61.3:9300}{dilm}{ml.machine_memory=135024914432, xpack.installed=true, ml.max_open_jobs=20}]}" }
{"type": "server", "timestamp": "2020-03-30T06:30:40,377Z", "level": "INFO", "component": "o.e.c.s.ClusterApplierService", "cluster.name": "es-docker-cluster", "node.name": "es01", "message": "master node changed {previous [], current [{es01}{TxAWx5BJRKWaFA4_n7dIDA}{OlRlsCl9SrCT9qNDlbIjSQ}{10.0.61.3}{10.0.61.3:9300}{dilm}{ml.machine_memory=135024914432, xpack.installed=true, ml.max_open_jobs=20}]}, term: 9, version: 40, reason: Publication{term=9, version=40}" }
{"type": "server", "timestamp": "2020-03-30T06:30:40,418Z", "level": "INFO", "component": "o.e.h.AbstractHttpServerTransport", "cluster.name": "es-docker-cluster", "node.name": "es01", "message": "publish_address {10.0.61.3:9200}, bound_addresses {0.0.0.0:9200}", "cluster.uuid": "_s5dqoIUSAyNPNUvb-TeBg", "node.id": "TxAWx5BJRKWaFA4_n7dIDA"  }
{"type": "server", "timestamp": "2020-03-30T06:30:40,418Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "es-docker-cluster", "node.name": "es01", "message": "started", "cluster.uuid": "_s5dqoIUSAyNPNUvb-TeBg", "node.id": "TxAWx5BJRKWaFA4_n7dIDA"  }
{"type": "server", "timestamp": "2020-03-30T06:30:40,627Z", "level": "INFO", "component": "o.e.l.LicenseService", "cluster.name": "es-docker-cluster", "node.name": "es01", "message": "license [2db2e78c-fd6f-4c49-b4a6-a230739bac5c] mode [basic] - valid", "cluster.uuid": "_s5dqoIUSAyNPNUvb-TeBg", "node.id": "TxAWx5BJRKWaFA4_n7dIDA"  }
{"type": "server", "timestamp": "2020-03-30T06:30:40,628Z", "level": "INFO", "component": "o.e.x.s.s.SecurityStatusChangeListener", "cluster.name": "es-docker-cluster", "node.name": "es01", "message": "Active license is now [BASIC]; Security is disabled", "cluster.uuid": "_s5dqoIUSAyNPNUvb-TeBg", "node.id": "TxAWx5BJRKWaFA4_n7dIDA"  }
{"type": "server", "timestamp": "2020-03-30T06:30:40,637Z", "level": "INFO", "component": "o.e.g.GatewayService", "cluster.name": "es-docker-cluster", "node.name": "es01", "message": "recovered [0] indices into cluster_state", "cluster.uuid": "_s5dqoIUSAyNPNUvb-TeBg", "node.id": "TxAWx5BJRKWaFA4_n7dIDA"  }

logs from node 2:

{"type": "server", "timestamp": "2020-03-30T06:29:00,459Z", "level": "DEBUG", "component": "o.e.a.ActionModule", "cluster.name": "es-docker-cluster", "node.name": "es02", "message": "Using REST wrapper from plugin org.elasticsearch.xpack.security.Security" }
{"type": "server", "timestamp": "2020-03-30T06:29:00,577Z", "level": "INFO", "component": "o.e.d.DiscoveryModule", "cluster.name": "es-docker-cluster", "node.name": "es02", "message": "using discovery type [zen] and seed hosts providers [settings]" }
{"type": "server", "timestamp": "2020-03-30T06:29:01,403Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "es-docker-cluster", "node.name": "es02", "message": "initialized" }
{"type": "server", "timestamp": "2020-03-30T06:29:01,403Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "es-docker-cluster", "node.name": "es02", "message": "starting ..." }
{"type": "server", "timestamp": "2020-03-30T06:29:01,525Z", "level": "INFO", "component": "o.e.t.TransportService", "cluster.name": "es-docker-cluster", "node.name": "es02", "message": "publish_address {10.0.60.12:9300}, bound_addresses {0.0.0.0:9300}" }
{"type": "server", "timestamp": "2020-03-30T06:29:01,789Z", "level": "INFO", "component": "o.e.b.BootstrapChecks", "cluster.name": "es-docker-cluster", "node.name": "es02", "message": "bound or publishing to a non-loopback address, enforcing bootstrap checks" }
ERROR: [2] bootstrap checks failed
[1]: memory locking requested for elasticsearch process but memory is not locked
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/es-docker-cluster.log
{"type": "server", "timestamp": "2020-03-30T06:29:01,827Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "es-docker-cluster", "node.name": "es02", "message": "stopping ..." }
{"type": "server", "timestamp": "2020-03-30T06:29:01,848Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "es-docker-cluster", "node.name": "es02", "message": "stopped" }
{"type": "server", "timestamp": "2020-03-30T06:29:01,848Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "es-docker-cluster", "node.name": "es02", "message": "closing ..." }
{"type": "server", "timestamp": "2020-03-30T06:29:01,864Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "es-docker-cluster", "node.name": "es02", "message": "closed" }
{"type": "server", "timestamp": "2020-03-30T06:29:01,881Z", "level": "INFO", "component": "o.e.x.m.p.NativeController", "cluster.name": "es-docker-cluster", "node.name": "es02", "message": "Native controller process has stopped - no new native processes can be started" }
7

Please read https://www.elastic.co/guide/en/elasticsearch/reference/current/bootstrap-checks.html

8

It also looks like you are using old cluster configuration setting that are no longer applicable in Elasticsearch 7.x, e.g. minimum_naster_nodes. Check the docs for examples of how to properly set up a Elasticsearch 7.x cluster.

9

Thanks for bringing this link up. have gone through this page couple of times from previous posts but still couldn't figure out the problem.
seems to be an issue with with network.host value. tried with all values suggested like site,local, globe and IP of the first container host nothing worked.

can you suggest what should be given for network.host for the second node to join the cluster.

10 
I have tried with elasticsearch 6.8.4 version and 7.6.1 version both have different environment set values where they are not applicable in recent versions. I noticed some observations.

`discovery.seed_hosts and discovery.zen.ping.unicast.hosts can't be used as same time

Even i tried commenting out minimum_master_nodes , still getting same error.
11

just a note. The same compose file working and 2 nodes joining the cluster when i start the swarm in same node.

i verified it with below command, I can see 2 nodes.

curl http://127.0.0.1:9200/_cluster/health?pretty

It is failing when i start 2 nodes in 2 seperate docker host servers.

12

can any one please help on this.

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.