Maximum number of data nodes per cluster

Elastic Stack Elasticsearch
1

We are running elasticsearch 2.4. One of our cluster has 123 data nodes and 5 master nodes.

The cluster is provisioned on AWS and we use the i2.2xlarge instance type (8 vcpu's, 61 GB of RAM, Up to 10 Gigabit of network bandwidth and 1.9 TB of SSD attached to the instance) for data node and m3.large instance type (2 vcpu's, 7.5 GB of RAM, moderate network performance, 32 GB SSD attached to the instance) for master node.

This cluster is write heavy (65% of IOPS are writes and 35% of IOPS are reads). Data is distributed across all nodes evenly. We don't see any performance bottlenecks on this cluster.

  1. How to determine the maximum no. of data nodes this cluster can support?
  2. Is there any best practice for the no. of data nodes that a cluster can have?

Please let me know if you require more details.

2

I'd suggest you are reaching the practical limit, even if there is no technical limit.

You should;

  1. Look to upgrade ASAP. 2.X is now entirely EOL
  2. Split the cluster into smaller logical groupings
  3. Use Cross Cluster Search to get a federated view of things
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.