# \[node-frozen\] fatal exception while booting Elasticsearchjava.io.UncheckedIOException: java.io.IOException: Not enough free space \[55991361536\] for cache file of size \[131852140544\] in path \[/usr/bin/elasticsearch-node-frozen/elasticsearch-8.15.1/data\] **URL:** \ **Category:** Elasticsearch\ **Created:** [October 1, 2024, 8:25am UTC](https://discuss.elastic.co/t/node-frozen-fatal-exception-while-booting-elasticsearchjava-io-uncheckedioexception-java-io-ioexception-not-enough-free-space-55991361536-for-cache-file-of-size-131852140544-in-path-usr-bin-elasticsearch-node-frozen-elasticsearch-8-15-1-data/368072 "2024-10-01T08:25:46Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![ax-va](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ax-va/32/137987_2.png) [@ax-va](https://discuss.elastic.co/u/ax-va)\ **Post date:** [October 1, 2024, 8:25am UTC](https://discuss.elastic.co/t/node-frozen-fatal-exception-while-booting-elasticsearchjava-io-uncheckedioexception-java-io-ioexception-not-enough-free-space-55991361536-for-cache-file-of-size-131852140544-in-path-usr-bin-elasticsearch-node-frozen-elasticsearch-8-15-1-data/368072/1 "2024-10-01T08:25:46Z")
I have a problem with adding a new frozen node. At first, I have installed Elasticsearch of version 8.15.1 on my local machine by extracting `tar.gz` on Ubuntu 22.04. LTS. It is working fine. Then, I have added a cold node (also extracted from same `.tar.gz`). It is also working fine. Finally, I have tried to add a frozen node (also extracted from same `.tar.gz`) with ```auto $ sudo -u elastic ./bin/elasticsearch --enrollment-token ``` but I have got an error during running at first time: ```auto [2024-09-30T21:29:08,851][ERROR][o.e.b.Elasticsearch] [node-frozen] fatal exception while booting Elasticsearchjava.io.UncheckedIOException: java.io.IOException: Not enough free space [55991361536] for cache file of size [131852140544] in path [/usr/bin/elasticsearch-node-frozen/elasticsearch-8.15.1/data] at org.elasticsearch.blobcache@8.15.1/org.elasticsearch.blobcache.shared.SharedBlobCacheService.(SharedBlobCacheService.java:378) at org.elasticsearch.blobcache@8.15.1/org.elasticsearch.blobcache.shared.SharedBlobCacheService.(SharedBlobCacheService.java:338) at org.elasticsearch.searchablesnapshots@8.15.1/org.elasticsearch.xpack.searchablesnapshots.SearchableSnapshots.createComponents(SearchableSnapshots.java:335) at org.elasticsearch.server@8.15.1/org.elasticsearch.node.NodeConstruction.lambda$construct$13(NodeConstruction.java:868) at org.elasticsearch.server@8.15.1/org.elasticsearch.plugins.PluginsService.lambda$flatMap$1(PluginsService.java:253) See logs for more details. ERROR: Elasticsearch did not exit normally - check the logs at /usr/bin/elasticsearch-node-frozen/elasticsearch-8.15.1/logs/elasticsearch.log ERROR: Elasticsearch died while starting up, with exit code 1 ``` The node has been not added. When I am trying to run it again with the same command ```auto $ sudo -u elastic ./bin/elasticsearch --enrollment-token ``` I am getting an error `ERROR: Skipping security auto configuration because it appears that the node is not starting up for the first time. The node might already be part of a cluster and this auto setup utility is designed to configure Security for new clusters only., with exit code 80`. My questions are: 1. Why does the frozen node need ca. 132 GB??? (I have only ca. 56 GB of free memory and I have no big data.) 2. How can I resolve the problem??? (I cannot increase hard disc memory on my machine.) What can I do to re-add the frozen node with resources I have? Thank you! ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 1, 2024, 8:57am UTC](https://discuss.elastic.co/t/node-frozen-fatal-exception-while-booting-elasticsearchjava-io-uncheckedioexception-java-io-ioexception-not-enough-free-space-55991361536-for-cache-file-of-size-131852140544-in-path-usr-bin-elasticsearch-node-frozen-elasticsearch-8-15-1-data/368072/2 "2024-10-01T08:57:07Z")
> [@ax-va](#): > > Why does the frozen node need ca. 132 GB??? It pre-allocates all the space it needs for its cache, which by default is 90% of the disk. See [these docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/searchable-snapshots.html) for more information, particularly about the `xpack.searchable.snapshot.shared_cache.size` setting. ---
**Author:** ![ax-va](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ax-va/32/137987_2.png) [@ax-va](https://discuss.elastic.co/u/ax-va)\ **Post date:** [October 1, 2024, 11:26am UTC](https://discuss.elastic.co/t/node-frozen-fatal-exception-while-booting-elasticsearchjava-io-uncheckedioexception-java-io-ioexception-not-enough-free-space-55991361536-for-cache-file-of-size-131852140544-in-path-usr-bin-elasticsearch-node-frozen-elasticsearch-8-15-1-data/368072/3 "2024-10-01T11:26:32Z")
I added to `elasticsearch.yml` of the frozen node two lines ```yaml xpack.searchable.snapshot.shared_cache.size: 8GB xpack.searchable.snapshot.shared_cache.size.max_headroom: 4GB ``` I don't need here too many storage for my learning. How can I add the node now? The command ```auto $ sudo -u elastic ./bin/elasticsearch --enrollment-token ``` gives me ```auto ERROR: Skipping security auto configuration because it appears that the node is not starting up for the first time. The node might already be part of a cluster and this auto setup utility is designed to configure Security for new clusters only., with exit code 80 ``` I think, I should remove this frozen node in the cluster to start it at the first time to re-add it to the cluster again... But I don't know how. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 1, 2024, 11:57am UTC](https://discuss.elastic.co/t/node-frozen-fatal-exception-while-booting-elasticsearchjava-io-uncheckedioexception-java-io-ioexception-not-enough-free-space-55991361536-for-cache-file-of-size-131852140544-in-path-usr-bin-elasticsearch-node-frozen-elasticsearch-8-15-1-data/368072/4 "2024-10-01T11:57:04Z")
Yes I think it'd be simplest to start again from scratch with this node - at least, delete the contents of its data path and revert the contents of its config path back to how they were when you unpacked it. ---
**Author:** ![ax-va](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ax-va/32/137987_2.png) [@ax-va](https://discuss.elastic.co/u/ax-va)\ **Post date:** [October 1, 2024, 1:52pm UTC](https://discuss.elastic.co/t/node-frozen-fatal-exception-while-booting-elasticsearchjava-io-uncheckedioexception-java-io-ioexception-not-enough-free-space-55991361536-for-cache-file-of-size-131852140544-in-path-usr-bin-elasticsearch-node-frozen-elasticsearch-8-15-1-data/368072/5 "2024-10-01T13:52:05Z")
When the cluster thinks that the node is added, how can I remove this node in the cluster properly? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 1, 2024, 2:03pm UTC](https://discuss.elastic.co/t/node-frozen-fatal-exception-while-booting-elasticsearchjava-io-uncheckedioexception-java-io-ioexception-not-enough-free-space-55991361536-for-cache-file-of-size-131852140544-in-path-usr-bin-elasticsearch-node-frozen-elasticsearch-8-15-1-data/368072/6 "2024-10-01T14:03:46Z")
I am not sure I understand the question. To remove a node from the cluster, just shut it down (i.e. kill the `elasticsearch` process). ---
**Author:** ![ax-va](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ax-va/32/137987_2.png) [@ax-va](https://discuss.elastic.co/u/ax-va)\ **Post date:** [October 1, 2024, 2:23pm UTC](https://discuss.elastic.co/t/node-frozen-fatal-exception-while-booting-elasticsearchjava-io-uncheckedioexception-java-io-ioexception-not-enough-free-space-55991361536-for-cache-file-of-size-131852140544-in-path-usr-bin-elasticsearch-node-frozen-elasticsearch-8-15-1-data/368072/7 "2024-10-01T14:23:57Z")
The problem is the `elasticsearch-node-frozen` node is not in the list of running processes obtained with `$ ps aux`. For example, the other `elasticsearch-node-cold` node is running and I can see its ID. And I do not know, how I can re-install or re-add the the `elasticsearch-node-frozen` node with the changed `elasticsearch.yml` setting. Please help. ---
**Author:** ![ax-va](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ax-va/32/137987_2.png) [@ax-va](https://discuss.elastic.co/u/ax-va)\ **Post date:** [October 1, 2024, 2:26pm UTC](https://discuss.elastic.co/t/node-frozen-fatal-exception-while-booting-elasticsearchjava-io-uncheckedioexception-java-io-ioexception-not-enough-free-space-55991361536-for-cache-file-of-size-131852140544-in-path-usr-bin-elasticsearch-node-frozen-elasticsearch-8-15-1-data/368072/8 "2024-10-01T14:26:40Z")
Or do you mean, I should kill the main elasticsearch process? ---
**Author:** ![ax-va](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ax-va/32/137987_2.png) [@ax-va](https://discuss.elastic.co/u/ax-va)\ **Post date:** [October 1, 2024, 2:39pm UTC](https://discuss.elastic.co/t/node-frozen-fatal-exception-while-booting-elasticsearchjava-io-uncheckedioexception-java-io-ioexception-not-enough-free-space-55991361536-for-cache-file-of-size-131852140544-in-path-usr-bin-elasticsearch-node-frozen-elasticsearch-8-15-1-data/368072/9 "2024-10-01T14:39:07Z")
I have now these running processes related to ES. ```auto root 6688 0.0 0.0 14980 1748 pts/0 S+ Sep27 0:00 sudo -u elastic ./bin/elasticsearch root 6689 0.0 0.0 14980 1668 pts/1 Ss Sep27 0:00 sudo -u elastic ./bin/elasticsearch elastic 6690 0.0 0.4 2610228 99368 pts/1 Sl+ Sep27 2:03 /usr/bin/elasticsearch/elasticsearch-8.15.1/jdk/bin/java -Xms4m -Xmx64m -XX:+UseSerialGC -Dcli.name=server -Dcli.script=./bin/elasticsearch -Dcli.libs=l elastic 6751 0.4 25.3 10380980 5032696 pts/1 Sl+ Sep27 21:39 /usr/bin/elasticsearch/elasticsearch-8.15.1/jdk/bin/java -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -Djava.security.mana elastic 6779 0.0 0.0 108652 860 pts/1 Sl+ Sep27 0:00 /usr/bin/elasticsearch/elasticsearch-8.15.1/modules/x-pack-ml/platform/linux-x86_64/bin/controller root 51832 0.0 0.0 14984 2864 pts/5 S+ Sep30 0:00 sudo -u elastic ./bin/elasticsearch --enrollment-token ... root 51833 0.0 0.0 14984 2260 pts/6 Ss Sep30 0:00 sudo -u elastic ./bin/elasticsearch --enrollment-token ... elastic 51834 0.0 0.7 2676804 147124 pts/6 Sl+ Sep30 0:36 /usr/bin/elasticsearch-node-cold/elasticsearch-8.15.1/jdk/bin/java -Xms4m -Xmx64m -XX:+UseSerialGC -Dcli.name=server -Dcli.script=./bin/elasticsearch -D elastic 51907 0.2 12.9 8080332 2579220 pts/6 Sl+ Sep30 3:06 /usr/bin/elasticsearch-node-cold/elasticsearch-8.15.1/jdk/bin/java -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -Djava.sec elastic 51937 0.0 0.0 108652 2340 pts/6 Sl+ Sep30 0:00 /usr/bin/elasticsearch-node-cold/elasticsearch-8.15.1/modules/x-pack-ml/platform/linux-x86_64/bin/controller ``` What should I kill? All? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 1, 2024, 3:27pm UTC](https://discuss.elastic.co/t/node-frozen-fatal-exception-while-booting-elasticsearchjava-io-uncheckedioexception-java-io-ioexception-not-enough-free-space-55991361536-for-cache-file-of-size-131852140544-in-path-usr-bin-elasticsearch-node-frozen-elasticsearch-8-15-1-data/368072/10 "2024-10-01T15:27:18Z")
Looks like you have two separate Elasticsearch nodes here. You can get the PIDs of the main node processes with `GET _cat/nodes?h=name,ip,pid&v` - that's the one to kill, it should clean up the other two.