Load balance in elasticsearch with multiple nodes in same machine

[surajbs (Suraj)]

Hi,

I am trying to run multiple nodes with different instances in same machine for load balancing in elasticsearch.

I have downloaded three elasticsearch files in the same machine and configured their elasticsearch.yml file with different settings. Please see below:

1. Elasticsearch - default configuration

2. Elasticsearch-node1
   cluster.name: Animals
   node.name: Snake
   node.master: true
   node.data: true
   transport.tcp.port: 9300
   http.port: 9200
   discovery.zen.ping.unicast.hosts: ["Snake", "Baboon"]

3. Elasticsearch-node2
   cluster.name: Animals
   node.name: Baboon
   node.master: false
   node.data: true
   transport.tcp.port: 9301
   http.port: 9201
   discovery.zen.ping.unicast.hosts: ["Snake", "Baboon"]

Please share your opinion and ideas on this.

Thank you.

[A_B]

You don't really need three different binaries (at least on Linux). You can start separate processes with different config files and data paths...

I would leave all as node.master: true. You are also using both for discovery.zen.ping.unicast.hosts:

Default is ports 9200/9300 so node1 (your second node) can't use those, bump them up with one for second and third node.

I'm running 5 nodes on same machine on 6 machines in total over two clusters. Definitely can be done

[surajbs (Suraj)]

Hi,

Thanks for the response.

I am able to create multiple instances with the nodes.

Please find my elasticsearch.yml config below.

• Elasticsearch - default configuration

• Elasticsearch-node1
  cluster.name: Animals
  node.name: Snake
  node.master: true
  node.data: true
  path.data: "C:\elasticsearch-5.4.2-node1\elasticsearch-5.4.2\data"
  path.logs: "C:\elasticsearch-5.4.2-node1\elasticsearch-5.4.2\logs"
  http.port: 9201
  discovery.zen.ping.unicast.hosts: ["127.0.0.1"]

• Elasticsearch-node2
  cluster.name: Animals
  node.name: Baboon
  node.master: false
  node.data: true
  path.data: "C:\elasticsearch-5.4.2-node2\elasticsearch-5.4.2\data"
  path.logs: "C:\elasticsearch-5.4.2-node2\elasticsearch-5.4.2\logs"
  http.port: 9202
  discovery.zen.ping.unicast.hosts: ["127.0.0.1"]

FYI, the status is red in the cluster health.

And I am using logstash to load the csv file into elasticsearch for which I have used the below config. But I am not able to see the other instances(9201 and 9202) loading the data.

output
{
elasticsearch {
hosts => ["localhost:9200", "localhost:9201", "localhost:9202"]
index => "cmss"
document_type => "cms"
document_id => "%{id}"
}
}

Thank you.

[surajbs (Suraj)]

Hi,

Thanks for the response.

I am able to create multiple instances with the nodes.

Please find my elasticsearch.yml config below.

Elasticsearch - default configuration

Elasticsearch-node1
cluster.name: Animals
node.name: Snake
node.master: true
node.data: true
path.data: "C:\elasticsearch-5.4.2-node1\elasticsearch-5.4.2\data"
path.logs: "C:\elasticsearch-5.4.2-node1\elasticsearch-5.4.2\logs"
http.port: 9201
discovery.zen.ping.unicast.hosts: ["127.0.0.1"]

Elasticsearch-node2
cluster.name: Animals
node.name: Baboon
node.master: false
node.data: true
path.data: "C:\elasticsearch-5.4.2-node2\elasticsearch-5.4.2\data"
path.logs: "C:\elasticsearch-5.4.2-node2\elasticsearch-5.4.2\logs"
http.port: 9202
discovery.zen.ping.unicast.hosts: ["127.0.0.1"]

FYI, the status is red in the cluster health.
And I am using logstash to load the csv file into elasticsearch for which I have used the below config. But I am not able to see the other instances(9201 and 9202) loading the data.

output
{
elasticsearch {
hosts => ["localhost:9200", "localhost:9201", "localhost:9202"]
index => "cmss"
document_type => "cms"
document_id => "%{id}"
}
}

Thank you.