We had more load on our cluster than usual and at that time we noticed some strange behavior. A node that hold primaries had much higher GC count. All nodes have the same configuration, JVM: 1.8.0_162 ES: 5.6.4. The only difference between node #1 and node #4 was that #1 had 2 primaries while #4 had 2 replicas. All shards have roughly the same size (~5GB).
Is it possible that node that holds primaries have higher memory consumption that one that holds only replicas?
Scripted updates are performed on the primary, and the resulting document is then replicated, meaning that the script processing is likely to result in more GC on the node with more primary shards.
Great, thx. That explains a lot.
Is there maybe a feature to "force" even distribution of primary shards across the nodes? In our case, I'd like to have primary for shard #3 on node #4. It doesn't have to be automatic, maybe we can add a feature in our service to watch for that kind of state and manually promote replica shards
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
