# How many Elasticsearch instances I should have in a node **URL:** \ **Category:** Elasticsearch\ **Created:** [May 10, 2019, 5:34pm UTC](https://discuss.elastic.co/t/how-many-elasticsearch-instances-i-should-have-in-a-node/180599 "2019-05-10T17:34:20Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![worapojc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/worapojc/32/42782_2.png) [@worapojc](https://discuss.elastic.co/u/worapojc)\ **Post date:** [May 10, 2019, 5:34pm UTC](https://discuss.elastic.co/t/how-many-elasticsearch-instances-i-should-have-in-a-node/180599/1 "2019-05-10T17:34:20Z")
Hi Elasticsearch Team, I'm using EC2 instance (i3.4xlarge) for hot-data node. - Memory 122Gb - Disk 2 x 1.9 NVMe SSD There are 2 ES instances on this node. I assign the 2 disks for both instances. I configure -Xmx31g -Xms31g for each instance. ![30%20AM](https://us1.discourse-cdn.com/elastic/original/3X/0/2/02812a510b42579a68a86e64ee827962be280d7d.png) From the picture, is it a good idea to add an instance with the same configuration? The memory usage would be around 90+GB. I also found GC overhead log. For example, [2019-05-10T18:34:51,377][INFO][o.e.m.j.JvmGcMonitorService] [10.49.112.143-hdata-node-1][gc][5454] overhead, spent [367ms] collecting in the last [1.1s] ![32%20AM](https://us1.discourse-cdn.com/elastic/original/3X/1/2/12ca4258002c53c51375e6cb79e1121d8ffc372c.png) Will this increase the performance and resolve the GC overhead issue? Thanks ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 13, 2019, 6:09am UTC](https://discuss.elastic.co/t/how-many-elasticsearch-instances-i-should-have-in-a-node/180599/2 "2019-05-13T06:09:46Z")
It is recommended to use at most 50% of RAM for heap which is roughly what you have got at the moment. Elasticsearch required off-heap memory and relies on the OS page cache for performance so I would not recommend adding any further node. Also, that log message does not necessarily indicate any GC problem. What does your heap usage look like over time? Do you have a nice saw-tooth pattern? ---
**Author:** ![worapojc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/worapojc/32/42782_2.png) [@worapojc](https://discuss.elastic.co/u/worapojc)\ **Post date:** [May 13, 2019, 10:04am UTC](https://discuss.elastic.co/t/how-many-elasticsearch-instances-i-should-have-in-a-node/180599/3 "2019-05-13T10:04:16Z")
Thanks @Christian_Dahlqvist. I was configured 31G for 2 nodes in a host. > 497 118705 1 99 May10 ? 15-14:47:27 /usr/bin/java -Xms31g -Xmx31g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -server -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch/hdata-node-1 -Des.distribution.flavor=default -Des.distribution.type=rpm -cp /usr/share/elasticsearch/lib/\* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/10.49.112.143-hdata-node-1/hdata-node-1\_elasticsearch.pid -d > 497 121129 1 99 May10 ? 12-15:16:05 /usr/bin/java -Xms31g -Xmx31g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -server -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch/hdata-node-0 -Des.distribution.flavor=default -Des.distribution.type=rpm -cp /usr/share/elasticsearch/lib/\* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/10.49.112.143-hdata-node-0/hdata-node-0\_ elasticsearch.pid -d I took the snapshot only first instance of this host The JVM Heap looks like the nice saw-tooth pattern. ![es_2_instances_jvm](https://us1.discourse-cdn.com/elastic/original/3X/0/3/035dd69de25f9964aaa6baab7e22b47b657408c7.png) The indexing time... ![es_2_instances_indexing](https://us1.discourse-cdn.com/elastic/original/3X/0/b/0b4d4901726bfef9594c75f6f77ad8871e874f04.png) At the right of **Indexing Time (ms)** graph, you will see that the value was growing rapidly because I added the third instance with the same configuration (heap size 31G) I think the increasing of the indexing time relates to the shard relocation when I added the new node. For the new node, the JVM still have a nice saw-tooth. ![es_3_instances_jvm](https://us1.discourse-cdn.com/elastic/original/3X/d/d/ddc9e8d5fac1535efac943c41d55dd4f01345f8e.png) If Elasticsearch required off-heap memory, what will be the better option? - 2 nodes with 31G heap size (50% of Memory) - 3 nodes with 24G heap size (59% of Memory) From JvmGcMonitorService log message, could you describe more details? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 10, 2019, 10:04am UTC](https://discuss.elastic.co/t/how-many-elasticsearch-instances-i-should-have-in-a-node/180599/4 "2019-06-10T10:04:17Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.