Error: master not discovered yet , Elasticsearch cluster using docker swarm with three nodes on three separate servers

Elastic Stack Elasticsearch
1

I want to set up an elasticsearch cluster using docker swarm with three nodes on three separate servers. But when I do the docker stack deploy command and the service comes up,

NaNtC
NaNtC1320×79 4.84 KB

but I get docker logs from the container, there are errors on all three.
node ---> ubuntu1

{"@timestamp":"2023-12-09T14:11:54.628Z", "log.level": "WARN", "message":"address [10.0.12.8:9300], node [null], requesting [false] discovery result: [es02][10.0.0.29:9300] connect_exception: Failed execution: io.netty.channel.ConnectTimeoutException: connection timed out: 10.0.0.29/10.0.0.29:9300: connection timed out: 10.0.0.29/10.0.0.29:9300", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es03][generic][T#4]","log.logger":"org.elasticsearch.discovery.PeerFinder","elasticsearch.node.name":"es03","elasticsearch.cluster.name":"my-cluster"}
{"@timestamp":"2023-12-09T14:11:58.273Z", "log.level": "WARN", "message":"master not discovered yet, this node has not previously joined a bootstrapped cluster, and this node must discover master-eligible nodes [es01, es02, es03] to bootstrap a cluster: have discovered [{es03}{58joxi_jQPqr8cd0eHxySg}{uIrh9xE3SNGyLtZc6DAUUA}{es03}{10.0.0.25}{10.0.0.25:9300}{cdfhilmrstw}{8.10.4}{7000099-8100499}]; discovery will continue using [10.0.12.5:9300, 10.0.12.8:9300] from hosts providers and [{es03}{58joxi_jQPqr8cd0eHxySg}{uIrh9xE3SNGyLtZc6DAUUA}{es03}{10.0.0.25}{10.0.0.25:9300}{cdfhilmrstw}{8.10.4}{7000099-8100499}] from last-known cluster state; node term 0, last-accepted version 0 in term 0; for troubleshooting guidance, see https://www.elastic.co/guide/en/elasticsearch/reference/8.10/discovery-troubleshooting.html", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es03][cluster_coordination][T#1]","log.logger":"org.elasticsearch.cluster.coordination.ClusterFormationFailureHelper","elasticsearch.node.name":"es03","elasticsearch.cluster.name":"my-cluster"}

node ---> ubuntu2

{"@timestamp":"2023-12-09T14:09:23.515Z", "log.level": "WARN", "message":"address [10.0.12.8:9300], node [null], requesting [false] discovery result: [es02][10.0.0.29:9300] connect_exception: Failed execution: io.netty.channel.ConnectTimeoutException: connection timed out: 10.0.0.29/10.0.0.29:9300: connection timed out: 10.0.0.29/10.0.0.29:9300", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es01][generic][T#3]","log.logger":"org.elasticsearch.discovery.PeerFinder","elasticsearch.node.name":"es01","elasticsearch.cluster.name":"my-cluster"}
{"@timestamp":"2023-12-09T14:09:26.306Z", "log.level": "WARN", "message":"master not discovered yet, this node has not previously joined a bootstrapped cluster, and this node must discover master-eligible nodes [es01, es02, es03] to bootstrap a cluster: have discovered [{es01}{hlJovsOSSKW7_XcF5Wcw1A}{93k4zmohQEaW1WfH7UjoRA}{es01}{10.0.0.27}{10.0.0.27:9300}{cdfhilmrstw}{8.10.4}{7000099-8100499}]; discovery will continue using [10.0.12.8:9300, 10.0.12.2:9300] from hosts providers and [{es01}{hlJovsOSSKW7_XcF5Wcw1A}{93k4zmohQEaW1WfH7UjoRA}{es01}{10.0.0.27}{10.0.0.27:9300}{cdfhilmrstw}{8.10.4}{7000099-8100499}] from last-known cluster state; node term 0, last-accepted version 0 in term 0; for troubleshooting guidance, see https://www.elastic.co/guide/en/elasticsearch/reference/8.10/discovery-troubleshooting.html", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es01][cluster_coordination][T#1]","log.logger":"org.elasticsearch.cluster.coordination.ClusterFormationFailureHelper","elasticsearch.node.name":"es01","elasticsearch.cluster.name":"my-cluster"}

node ---> ubuntu3

{"@timestamp":"2023-12-09T14:10:26.640Z", "log.level": "WARN", "message":"address [10.0.12.2:9300], node [null], requesting [false] discovery result: [es03][10.0.0.25:9300] connect_timeout[30s]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es02][generic][T#3]","log.logger":"org.elasticsearch.discovery.PeerFinder","elasticsearch.node.name":"es02","elasticsearch.cluster.name":"my-cluster"}
{"@timestamp":"2023-12-09T14:10:30.387Z", "log.level": "WARN", "message":"master not discovered yet, this node has not previously joined a bootstrapped cluster, and this node must discover master-eligible nodes [es01, es02, es03] to bootstrap a cluster: have discovered [{es02}{SdcA-EyrRyujky7n4MliiA}{cwcUzo9WQOGq1so0di2lvQ}{es02}{10.0.0.29}{10.0.0.29:9300}{cdfhilmrstw}{8.10.4}{7000099-8100499}]; discovery will continue using [10.0.12.5:9300, 10.0.12.2:9300] from hosts providers and [{es02}{SdcA-EyrRyujky7n4MliiA}{cwcUzo9WQOGq1so0di2lvQ}{es02}{10.0.0.29}{10.0.0.29:9300}{cdfhilmrstw}{8.10.4}{7000099-8100499}] from last-known cluster state; node term 0, last-accepted version 0 in term 0; for troubleshooting guidance, see https://www.elastic.co/guide/en/elasticsearch/reference/8.10/discovery-troubleshooting.html", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es02][cluster_coordination][T#1]","log.logger":"org.elasticsearch.cluster.coordination.ClusterFormationFailureHelper","elasticsearch.node.name":"es02","elasticsearch.cluster.name":"my-cluster"}

The configuration file is as follows

version: '3.8'

services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.10.4
    container_name: es01
    environment:
      - node.name=es01
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - cluster.name=my-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - xpack.security.enabled=false
      - network.host=0.0.0.0
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata01:/usr/share/elasticsearch/data
    ports:
      - 9201:9200
    networks:
      - esnet

  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.10.4
    container_name: es02
    environment:
      - node.name=es02
      - discovery.seed_hosts=es01,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - cluster.name=my-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - xpack.security.enabled=false
      - network.host=0.0.0.0
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata02:/usr/share/elasticsearch/data
    ports:
      - 9202:9200
    networks:
      - esnet

  es03:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.10.4
    container_name: es03
    environment:
      - node.name=es03
      - discovery.seed_hosts=es01,es02
      - cluster.initial_master_nodes=es01,es02,es03
      - cluster.name=my-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - xpack.security.enabled=false
      - network.host=0.0.0.0
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata03:/usr/share/elasticsearch/data
    ports:
      - 9203:9200
    networks:
      - esnet

volumes:
  esdata01:
    driver: local
  esdata02:
    driver: local
  esdata03:
    driver: local
networks:
  esnet:

Thank you for helping me

2

The message ends with the following:

I'd suggest going there first.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.