# Cannot recover kibana indices due to 'Too many open files' error **URL:** \ **Category:** Elasticsearch\ **Created:** [May 28, 2024, 9:26am UTC](https://discuss.elastic.co/t/cannot-recover-kibana-indices-due-to-too-many-open-files-error/360365 "2024-05-28T09:26:42Z")\ **Posts on this page:** 1\ **Page:** 1
**Author:** ![111407](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111407/32/78114_2.png) [@111407](https://discuss.elastic.co/u/111407)\ **Post date:** [May 28, 2024, 9:26am UTC](https://discuss.elastic.co/t/cannot-recover-kibana-indices-due-to-too-many-open-files-error/360365/1 "2024-05-28T09:26:42Z")
After cluster reboot kibana indices cannot be recovered, I see the following entries in log: ```auto [2024-05-28T05:15:49,289][DEBUG][o.e.g.G.InternalPrimaryShardAllocator] [sd-07a1-ae2b.local] [[.kibana_2/4sWnjATIQ7WDrOTqWYr7fA]][0]: found 0 allocation candidates of [.kibana_2][0], node[null], [P], recovery_source[existing store recovery; bootstrap_history_uuid=false], s[UNASSIGNED], unassigned_info[[reason=MANUAL_ALLOCATION], at[2024-05-26T00:42:05.578Z], delayed=false, details[failed shard on node [4YnGQHt5SvKcQpdtROrImA]: failed recovery, failure RecoveryFailedException[[.kibana_2][0]: Recovery failed on {sd-c3d2-b7ca.local}{4YnGQHt5SvKcQpdtROrImA}{LaA75lNsQgac6B-x627_Uw}{sd-c3d2-b7ca.local}{10.115.205.88:9300}{ml.machine_memory=67387195392, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}]; nested: IndexShardRecoveryException[failed to recover from gateway]; nested: FileSystemException[/opt/gicapods/elasticdata/nodes/0/indices/4sWnjATIQ7WDrOTqWYr7fA/0/_state: **Too many open files**]; ], allocation_status[no_valid_shard_copy]] based on allocation ids: [[DmpGyU8ZSE6sGJJrEzRelw]] [2024-05-28T05:15:49,289][DEBUG][o.e.g.G.InternalPrimaryShardAllocator] [sd-07a1-ae2b.local] [[.kibana_2/4sWnjATIQ7WDrOTqWYr7fA]][0]: not allocating, number_of_allocated_shards_found [0] ``` I've increased open files limit to 1 million in `limits.conf` and applied the same to elasticsearch process via `prlimit` command. `lsof` shows me that ~300k files are currently opened by elasticsearch process, however I'm still seeing the above message in log. Is this issue really has something to do with open files? 2 nodes cluster, version 6.8