# File Descriptors leak on upgrade to 8.17.3

**URL:** <https://discuss.elastic.co/t/file-descriptors-leak-on-upgrade-to-8-17-3/376685>\
**Category:** Elasticsearch\
**Tags:** docker\
**Created:** [April 2, 2025, 1:19pm UTC](https://discuss.elastic.co/t/file-descriptors-leak-on-upgrade-to-8-17-3/376685 "2025-04-02T13:19:18Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ricardo\_Ferreira](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ricardo_ferreira/32/142405_2.png) [@Ricardo\_Ferreira](https://discuss.elastic.co/u/Ricardo_Ferreira)\
**Post date:** [April 2, 2025, 1:19pm UTC](https://discuss.elastic.co/t/file-descriptors-leak-on-upgrade-to-8-17-3/376685/1 "2025-04-02T13:19:18Z")

</div>

Hello,

I've tried searching around the forum for potential similar issues but couldn't find anything related so apologies in advance if this issue has been raised before.

I've completed an upgrade of our Elasticsearch cluster from version 8.10.2 to 8.17.3 and started receiving alerts for increased number of file descriptors being used.

When looking deeper into the issue we noticed that all these files appear to be deleted `.dvd` files that no longer exist but are still open.

when running `lsof -a -p 2356841 -d ^mem | wc -l` we get the following:

```auto
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 2349214 1000 cwd DIR 0,45 4096 539905 /usr/share/elasticsearch
java 2349214 1000 rtd DIR 0,45 4096 524308 /
java 2349214 1000 txt REG 0,45 12328 538322 /usr/share/elasticsearch/jdk/bin/java
java 2349214 1000 DEL REG 9,0 188875029 /usr/share/elasticsearch/data/indices/fu2Qku6HQd-7hidVDZrFXw/2/index/_59vj_22qk_Lucene90_0.dvd
java 2349214 1000 DEL REG 9,0 166462790 /usr/share/elasticsearch/data/indices/fmt6yzMqQyeAMyAq8PLLLw/1/index/_bjmj_3d4_Lucene90_0.dvd
java 2349214 1000 DEL REG 9,0 130810117 /usr/share/elasticsearch/data/indices/QOyV3F2ZT4Wrkw2txuSO5A/0/index/_2ecro_6ft_Lucene90_0.dvd
java 2349214 1000 DEL REG 9,0 130810172 /usr/share/elasticsearch/data/indices/QOyV3F2ZT4Wrkw2txuSO5A/0/index/_2ecro_6fs_Lucene90_0.dvd
java 2349214 1000 DEL REG 9,0 130809928 /usr/share/elasticsearch/data/indices/QOyV3F2ZT4Wrkw2txuSO5A/0/index/_2ecro_6fr_Lucene90_0.dvd
...
24400

```

We basically get an endless list of `DEL` FDs that appear to be files that have been deleted but are still open.

Has anyone ran into a similar issue? Anything I might be overlooking here?

Best regards,  
Ricardo Ferreira

---

<div class="post-metadata">

**Author:** ![Ricardo\_Ferreira](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ricardo_ferreira/32/142405_2.png) [@Ricardo\_Ferreira](https://discuss.elastic.co/u/Ricardo_Ferreira)\
**Post date:** [April 3, 2025, 8:34am UTC](https://discuss.elastic.co/t/file-descriptors-leak-on-upgrade-to-8-17-3/376685/2 "2025-04-03T08:34:14Z")

</div>

UPDATE:

After some hours the number of open files eventually started dropping and stabilized.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/f/4f757b207e349e794fbcb733bef3982980706743.png)

The image shows the `node_filefd_allocated` metric from node-exporter measured in all data nodes of the cluster.

After the upgrade we see a big increase, after a few hours we decided to restart the nodes one by one after we increased the `fs.file-max`. The growing number of open files eventually subsided and stabilized around 80,000.

---

<div class="post-metadata">

**Author:** ![Johan\_Nilsson\_Hansen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johan_nilsson_hansen/32/77950_2.png) [@Johan\_Nilsson\_Hansen](https://discuss.elastic.co/u/Johan_Nilsson_Hansen)\
**Post date:** [May 19, 2025, 8:38am UTC](https://discuss.elastic.co/t/file-descriptors-leak-on-upgrade-to-8-17-3/376685/3 "2025-05-19T08:38:46Z")

</div>

Hey @Ricardo_Ferreira . We also have the exact same pattern. We are upgrading from ES7 to 8.17.x.

We just found this issue [Elasticsearch 8.16.x Large Increase in MMAP Counts · Issue #119652 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/issues/119652) which seems to describe the pattern we have. It links this suggested fix which we will try out now [Performance degradation after upgrading from 8.6.1 to 8.16.1 · Issue #118623 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/issues/118623#issuecomment-2622802248)

---

<div class="post-metadata">

**Author:** ![Johan\_Nilsson\_Hansen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johan_nilsson_hansen/32/77950_2.png) [@Johan\_Nilsson\_Hansen](https://discuss.elastic.co/u/Johan_Nilsson_Hansen)\
**Post date:** [May 19, 2025, 11:48am UTC](https://discuss.elastic.co/t/file-descriptors-leak-on-upgrade-to-8-17-3/376685/4 "2025-05-19T11:48:39Z")

</div>

For anyone else that has this issue, the proposed solution [here](https://github.com/elastic/elasticsearch/issues/118623#issuecomment-2622802248) to set `-Dorg.apache.lucene.store.MMapDirectory.sharedArenaMaxPermits=1` worked for us.
