Elasticsearch voter_only instance is taking 50% memory usage

[MCRBEm]

Hello everyone.
Need some help to understand high memory issue we have and possibly fixed/improve our elastic stack setup (without adding new server). We were given only 2 servers to install elasticsearch and we must set it up in high availability. Each server has 32GB RAM and 16CPU and we do not have Kibana and Logtash running in these servers.

ServerA:
ES instance1 node.roles: [ data, master ]
ES instance2 node.roles: [ master, voting_only ]

ServerB:
ES instance3 node.roles: [ data, master ]
ES instance4 node.roles: [ master, voting_only ]

( total of 4 elasticsearch instances in the cluster)

We decided to have both master-data and master-voter_only in one server so we can avoid cluster going down when one server is down. But we are observing high memory utilization from voter_only instances.
If both instances (master-data and master-voter_only) are enabled in a server, the memory utilization gets 93%. If we turned off master-voter_only instance, the memory utilization becomes 40%. Any idea why voter_only instance is taking 50% memory usage? We tried changing the JVM Heap size in jvm.options file to 4GB or 8GB but it did not improve anything.

[Christian_Dahlqvist]

I would recommend you read this section of the docs. In you scenario you should only have one voting-only master node and the best you can achieve with only 2 hosts/availability zones available is resiliency to losing a single node. In order to be HA with respect to losing a full host you will need at least 3 hosts/availability zones.

[MCRBEm]

thank you. will try to get third server for master role and retry HA without voting_only role on the other 2 servers