Networking

Elastic Stack Elasticsearch
1

I have a 26 node cluster where 3 nodes have a public facing interface. My thought was to set up those 3 nodes as master/ingest nodes and the remaining nodes as data nodes. I have a 100Gb InfiniBand network that I would like to use for the node-to-node communications. From what I'm reading I should set the following:

network.publish_host: ib0:ipv4 -> IB network interface on the data nodes and master/ingest nodes
transport.tcp.port: 9300-9400 -> IB network port range for cluster communications
network.bind_host: eth0:ipv4 -> public interface on master/ingest nodes
http.port: 9200 -> port for public interfaces

My next question is if the parameters on the command line i.e. -Ehttp.port:9201 would override the value in the elasticsearch.yml file.

2

For a cluster that size I would recommend using dedicated master nodes that do not hold data nor serve traffic. This will increase stability and resilience as the dedicated master nodes are unlikely to suffer from long GC. Having master nodes double as coordinating and/or ingest nodes is in my opinion a bad idea at that scale.

3

Yes. It will.

4

For the master nodes I have dual 20-core CPUs and a 512Gb of memory so just having them as master nodes only would be a waste of resources, I've currently done some stress testing and have been able to maintain around 500,000 docs/sec at a rate of about 11Tb/hour with very little load on the master/ingest nodes. Just trying to use what was given to me. The data nodes have dual 14-care CPUs with the same 512Gb of memory and 6 1.6Tb SSDs - to make use of memory I've set up 4 ES data instances on each node giving them 6 cores each and 32Gb of memory (I'm still tuning this part).

5

You can always set up a dedicated master node as well as a separate coordinating/ingest node on those machines. Just having the dedicated master node in a separate JVM process without serving traffic will be beneficial.

6

that is the configuration I have

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.