Help in Creating a Cluster

[ryh]

Hi All,

I'm trying to create an elasticsearch cluster but wasn't sure how to go about it.

Here's a brief overview of our stack.

SERVER 1 - runs a python app with elasticsearch in the background
SERVER 2 - ELK stack

What should be the configuration for server 1 and server 2 so that I'm able to cluster them properly?

Thanks in advance,
Ry

[jakelandis]

If you are just getting started with Elasticsearch, I would suggest to start with Docker or Docker compose (or Elastic Cloud). Full Docker examples can be found here: Install Elasticsearch with Docker | Elasticsearch Guide [8.11] | Elastic

Logstash and Kibana also have docker images you can use to setup a full ELK stack.

Also, here is the python client library: Python Elasticsearch Client — Elasticsearch 8.0.0 documentation

What should be the configuration for server 1 and server 2 so that I'm able to cluster them properly?

The cluster settings to form a cluster can be found here for 7x: https://www.elastic.co/guide/en/elasticsearch/reference/current/discovery-settings.html

... and here for 6.x : Discovery settings | Elasticsearch Guide [6.8] | Elastic (they are similar, but not identical)

[ryh]

Thanks for the reply.

Unfortunately, I cannot use docker-compose because we don't have a separate container just for an elasticsearch per application.

Our goal is to have an application and an elasticsearch installed in one container.

Do you have any idea how to locally build a cluster? I ran a custom image with elasticsearch in it by doing the following: sudo docker run -e -i -t -v /opt/dicts/spitz/latest:/opt/dicts/spitz/latest --net=host gsw/mimidae:latest and an image sudo docker run -p 5601:5601 -p 9201:9200 -p 5044:5044 -it -e LOGSTASH_START=0 -e KIBANA_START=0 --name elk sebp/elk

If I removed the option --net=host and replaced it with the options -p 9200:9200 -p 9300:9300, I'm unable to curl the container's port. Same issue with my elk container.

Thanks,
Ry

[jakelandis]

If you want to cluster via docker run this should work (7x) :

docker network create my-net && 
docker run -d -p 9200:9200 --name es1 --network my-net \
            -e "node.name=es1" \
            -e "discovery.seed_hosts=es1,es2" \
            -e "cluster.initial_master_nodes=es1,es2" \
                 elastic/elasticsearch:7.1.1 &&
docker run -d -p 9201:9200 --name es2 --network my-net \
            -e "node.name=es2" \
            -e "discovery.seed_hosts=es1,es2" \
            -e "cluster.initial_master_nodes=es1,es2" \
                 elastic/elasticsearch:7.1.1   &&
watch -n 5 curl -v localhost:9200/_cat/nodes                 

• note any other docker containers need to use same network, or route requests through the host

[ryh]

Thanks.