# How to modify total memory of existing nodes?

**URL:** <https://discuss.elastic.co/t/how-to-modify-total-memory-of-existing-nodes/349958>\
**Category:** Elasticsearch\
**Tags:** docker\
**Created:** [December 26, 2023, 12:51pm UTC](https://discuss.elastic.co/t/how-to-modify-total-memory-of-existing-nodes/349958 "2023-12-26T12:51:57Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![SanthoshKMurugadass](https://avatars.discourse-cdn.com/v4/letter/s/b9e5f3/32.png) [@SanthoshKMurugadass](https://discuss.elastic.co/u/SanthoshKMurugadass)\
**Post date:** [December 26, 2023, 12:51pm UTC](https://discuss.elastic.co/t/how-to-modify-total-memory-of-existing-nodes/349958/1 "2023-12-26T12:51:58Z")

</div>

Hi,

I am new to elasticresearch. I have set up 3 node cluster following docker setup procedure explained as per section: **Section Start a multi-node cluster with Docker Compose** in below help link

> **[Install Elasticsearch with Docker | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#docker-compose-file)**

Here is my cluster node infromation.  
n id ramMax hm hp diskTotal heapCurrent  
es03 Fozz 3.7gb 1.8gb 43 442.7gb 843.9mb  
es01 fMbk 3.7gb 1.8gb 77 442.7gb 1.4gb  
es02 TdIO 3.7gb 1.8gb 22 442.7gb 438.9mb

The default total memory for each node is ~ 3.7 GB, i want to increase this to 32GB for each node to deploy ML Pretrained models. How can i do this ? I am new to elastic please provide step-by-step instructions.

Regards  
Santhosh M

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 26, 2023, 4:17pm UTC](https://discuss.elastic.co/t/how-to-modify-total-memory-of-existing-nodes/349958/2 "2023-12-26T16:17:07Z")

</div>

First, you do not want to set 32 GB as that will go beyond the java compressed object pointers and actually be less efficient. I would suggest setting it to 28 GB.... That is a safe number.

The jvm memory size is set in the .env file

> Download and save the following files in the project directory:
> 
> .env

To be clear, you will need all that memory available at startup or the nodes will fail.

Set the setting in the `.env`

Then  
`docker compose down`  
then  
`docker compose up -d`  
should work.

That would save the data and then create the new containers with the new memory size but still have the data / indices you already created because the volument mounts will not get destroyed.

Be careful 🙂

---

<div class="post-metadata">

**Author:** ![SanthoshKMurugadass](https://avatars.discourse-cdn.com/v4/letter/s/b9e5f3/32.png) [@SanthoshKMurugadass](https://discuss.elastic.co/u/SanthoshKMurugadass)\
**Post date:** [December 27, 2023, 8:05am UTC](https://discuss.elastic.co/t/how-to-modify-total-memory-of-existing-nodes/349958/3 "2023-12-27T08:05:06Z")

</div>

> [@stephenb](#):
>
> project

Dear @stephenb

Thanks for the suggestions. Unfortunately there is no option to download the .env file that you wanted to share. May i please request you to upload this file again?

---

<div class="post-metadata">

**Author:** ![SanthoshKMurugadass](https://avatars.discourse-cdn.com/v4/letter/s/b9e5f3/32.png) [@SanthoshKMurugadass](https://discuss.elastic.co/u/SanthoshKMurugadass)\
**Post date:** [December 27, 2023, 8:14am UTC](https://discuss.elastic.co/t/how-to-modify-total-memory-of-existing-nodes/349958/4 "2023-12-27T08:14:18Z")

</div>

I already have an .env file as below.  
What is the line that i need to add in crease the memory to 28GB ?

```auto
# Version of Elastic products
STACK_VERSION=8.11.1

# Set the cluster name
CLUSTER_NAME=docker-cluster

# Port to expose Elasticsearch HTTP API to the host
ES_PORT=9200
#ES_PORT=127.0.0.1:9200

# Port to expose Kibana to the host
KIBANA_PORT=5601
#KIBANA_PORT=80

# Increase or decrease based on the available host memory (in bytes)
MEM_LIMIT=4073741824

# Project namespace (defaults to the current folder name if not set)
#COMPOSE_PROJECT_NAME=myproject

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 27, 2023, 3:50pm UTC](https://discuss.elastic.co/t/how-to-modify-total-memory-of-existing-nodes/349958/5 "2023-12-27T15:50:52Z")

</div>

> [@SanthoshKMurugadass](#):
>
> ```auto
> # Increase or decrease based on the available host memory (in bytes)
> MEM_LIMIT=4073741824
> 
> ```

Looking closer you will set MEM\_LIMIT to the size of the container RAM for the elasticsearch node. The MAX Best practice is 64GB, and then Elasticsearch will compute the optimmum JVM Heap size. That is good. So Set `MEM_LIMIT` to 64GB.

`64GB = 64*1024^3 = 68719476736`

`MEM_LIMIT=68719476736`

Warning if you are using the 3 node Docker Cluster you will need at least 196GB of RAM on that host plus extra from OS and Kibana etc....

Also that same setting is used for Kibana which is not needed so in the `docker-compose.yml` I would just set the Kibana `mem_limit` to say 4GB

Or create a new variable in the `.env` file like

`KB_MEM_LIMIT= 4073741824`

and in the kibana section of the compose

` mem_limit: ${KB_MEM_LIMIT}`

---

<div class="post-metadata">

**Author:** ![SanthoshKMurugadass](https://avatars.discourse-cdn.com/v4/letter/s/b9e5f3/32.png) [@SanthoshKMurugadass](https://discuss.elastic.co/u/SanthoshKMurugadass)\
**Post date:** [January 4, 2024, 12:00pm UTC](https://discuss.elastic.co/t/how-to-modify-total-memory-of-existing-nodes/349958/6 "2024-01-04T12:00:53Z")

</div>

Hi Stephen Brown,

I could increase the total memory for each node by modifying the below line in .env file used to setup my docker container

Open .env file from /nfs/ ElasticData\_1

Change below lines and save the file

# Increase or decrease based on the available host memory (in bytes)

#MEM\_LIMIT=4073741824 # working

MEM\_LIMIT=64g

Shot down docker

$ docker-compose down # memory change not reflecting with docker-compose stop

Bring up Docker

$ docker-compose up -d # Not working with Stop & Start docker-compose start

Memory change not reflecting with docker-compose stop & Start docker-compose start . Need to shutdown and bring up as above.

Check the change by executing below command in Kibana Developer Console

GET /\_cat/nodes?v=true&h=n,id,ramMax,hm,hp,diskTotal,heapCurrent

n id ramMax hm hp diskTotal heapCurrent

es03 Fozz 64gb 31gb 12 442.7gb 3.7gb

es01 fMbk 64gb 31gb 8 442.7gb 2.5gb

es02 TdIO 64gb 31gb 1 442.7gb 326.8mb

I have a follow up question.

The pre-trained model I deployed is size of 4.1GB, when I tried setting my total memory of each node 12GB (before setting 64GB, I tried with 12GB/node), it failed to deploy with error shown in below screenshot, why ?

 ![](https://us1.discourse-cdn.com/elastic/original/3X/5/0/50f8e763e90f65b9d55f341124bfe8dcad4096e7.png)

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [January 4, 2024, 2:33pm UTC](https://discuss.elastic.co/t/how-to-modify-total-memory-of-existing-nodes/349958/7 "2024-01-04T14:33:00Z")

</div>

I think the message is pretty self-explanatory. In the message, there was only 3.5GB free and the model needed 4GB

 ![Screenshot 2024-01-04 at 6.30.59 AM](https://us1.discourse-cdn.com/elastic/original/3X/d/7/d740e794c241a37f34e1ff22732b696dcb6ebc09.jpeg)

---

<div class="post-metadata">

**Author:** ![SanthoshKMurugadass](https://avatars.discourse-cdn.com/v4/letter/s/b9e5f3/32.png) [@SanthoshKMurugadass](https://discuss.elastic.co/u/SanthoshKMurugadass)\
**Post date:** [January 4, 2024, 3:21pm UTC](https://discuss.elastic.co/t/how-to-modify-total-memory-of-existing-nodes/349958/8 "2024-01-04T15:21:36Z")

</div>

Yes, understand the error message. But when i allocated 12gb why only 3.5 GB is free. Where did remaining go ? Is there anyway to check split up of how 12gb is allocated to different purposes?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [January 4, 2024, 3:51pm UTC](https://discuss.elastic.co/t/how-to-modify-total-memory-of-existing-nodes/349958/9 "2024-01-04T15:51:32Z")

</div>

So when you allocate 12GB to a node 50% is allocated to JVM so that is only 6GB for the JVM.

Then elasticsearch uses the JVM memory for many functions you can read about what items consume memory

If you really want to know read this page in detail.

> **[Size your shards | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/size-your-shards.html#each-shard-has-overhead)**

And

> **[Size your shards | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/size-your-shards.html#field-count-recommendation)**

> **[Size your shards | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/size-your-shards.html#_retrieving_heap_size_and_field_mapper_overheads)**

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 1, 2024, 3:51pm UTC](https://discuss.elastic.co/t/how-to-modify-total-memory-of-existing-nodes/349958/10 "2024-02-01T15:51:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.