# Problem with index folders in ES 2.3.2 **URL:** \ **Category:** Elasticsearch\ **Created:** [May 17, 2016, 10:27am UTC](https://discuss.elastic.co/t/problem-with-index-folders-in-es-2-3-2/50209 "2016-05-17T10:27:31Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Jose\_A\_Garcia](https://avatars.discourse-cdn.com/v4/letter/j/f19dbf/32.png) [@Jose\_A\_Garcia](https://discuss.elastic.co/u/Jose_A_Garcia)\ **Post date:** [May 17, 2016, 10:27am UTC](https://discuss.elastic.co/t/problem-with-index-folders-in-es-2-3-2/50209/1 "2016-05-17T10:27:32Z")
Hi, We are migrating from an old elasticsearch version (1.4) to the latest (2.3.2). We are doing some tests and I'm having some problems. Our deployment is a bit unconventional, we have several indexes, each one using a different mount point (when we create a new index we also create a mount point with the same name in the data folder, so that ES can store the data in there). The mount points are local to the server, not remote storage. This works quite well in ES 1.4, but in the latest version we have issues when we try to delete an index. When we send the delete call to elasticsearch the index folder is still mounted, so it cannot be deleted (you get a resource busy error). In the old version it didn't print any errors, it just deleted the contents, leaving the folder, and continued. With the latest we get several exceptions: java.nio.file.FileSystemException: Device or resource busy (which was expected) and: LockObtainFailedException[Can't lock shard [index-1][0], timed out after 5000ms]; at org.elasticsearch.index.IndexService.createShard(IndexService.java:389) at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyInitializingShard(IndicesClusterStateService.java:601) at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyNewOrUpdatedShards(IndicesClusterStateService.java:501) at org.elasticsearch.indices.cluster.IndicesClusterStateService.clusterChanged(IndicesClusterStateService.java:166) at org.elasticsearch.cluster.service.InternalClusterService.runTasksForExecutor(InternalClusterService.java:610) at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:772) at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231) at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) After this the cluster goes into a Red state and everything stops working. I there anything we can do to solve this? Maybe an older version that keeps the old behaviour? Thanks! ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 17, 2016, 2:20pm UTC](https://discuss.elastic.co/t/problem-with-index-folders-in-es-2-3-2/50209/2 "2016-05-17T14:20:58Z")
> [@Jose\_A\_Garcia](#): > > Our deployment is a bit unconventional, we have several indexes, each one using a different mount point (when we create a new index we also create a mount point with the same name in the data folder, so that ES can store the data in there). The mount points are local to the server, not remote storage. How do you even do this? ES cannot store a specific index on a specific mount/ ---
**Author:** ![Jose\_A\_Garcia](https://avatars.discourse-cdn.com/v4/letter/j/f19dbf/32.png) [@Jose\_A\_Garcia](https://discuss.elastic.co/u/Jose_A_Garcia)\ **Post date:** [May 17, 2016, 2:34pm UTC](https://discuss.elastic.co/t/problem-with-index-folders-in-es-2-3-2/50209/3 "2016-05-17T14:34:51Z")
Before creating the index we create a mount in the data folder (.../data/index1), then we create the index in ES and it just writes to the mounted folder in that path... ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 17, 2016, 2:42pm UTC](https://discuss.elastic.co/t/problem-with-index-folders-in-es-2-3-2/50209/4 "2016-05-17T14:42:56Z")
Ok, this is a really bad idea. \*Really , **really** bad. You cannot guarantee that ES will only write that index to that mount. ---
**Author:** ![Jose\_A\_Garcia](https://avatars.discourse-cdn.com/v4/letter/j/f19dbf/32.png) [@Jose\_A\_Garcia](https://discuss.elastic.co/u/Jose_A_Garcia)\ **Post date:** [May 17, 2016, 2:49pm UTC](https://discuss.elastic.co/t/problem-with-index-folders-in-es-2-3-2/50209/5 "2016-05-17T14:49:24Z")
So that means that all the data for a particular index is not guaranteed to be under the same folder in a node? For what I've seen it creates a folder with the name of the index in the data directory and stores everything inside. At least in the old versions. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 17, 2016, 2:49pm UTC](https://discuss.elastic.co/t/problem-with-index-folders-in-es-2-3-2/50209/6 "2016-05-17T14:49:51Z")
> [@Jose\_A\_Garcia](#): > > So that means that all the data for a particular index is not guaranteed to be under the same folder in a node? Yes. > [@Jose\_A\_Garcia](#): > > For what I've seen it creates a folder with the name of the index in the data directory and stores everything inside. At least in the old versions. This has never been guaranteed behaviour. ---
**Author:** ![Jose\_A\_Garcia](https://avatars.discourse-cdn.com/v4/letter/j/f19dbf/32.png) [@Jose\_A\_Garcia](https://discuss.elastic.co/u/Jose_A_Garcia)\ **Post date:** [May 17, 2016, 3:25pm UTC](https://discuss.elastic.co/t/problem-with-index-folders-in-es-2-3-2/50209/7 "2016-05-17T15:25:33Z")
Looking at this link: > **[A Dive into the Elasticsearch Storage | Elastic](https://www.elastic.co/blog/found-dive-into-elasticsearch-storage)** > > In this article we'll investigate the files written to the data directory by various parts of Elasticsearch. We will look at node, index and shard level files and give a short explanation of their contents in order to establish an understanding of... It looks like it should write everything (excluding snapshots) to the same folder, data/elasticsearch/nodes/0/indices/foo. I realize that this is not guaranteed officially, but it's been doing it in our cluster for quite some time too... Still, the issue we have shouldn't be related to that, in never tries to write anywhere else than where we expect. The problem comes when we delete the index. After failing to delete the folder the whole cluster breaks down and complains about not being able to obtain the lock. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 17, 2016, 3:53pm UTC](https://discuss.elastic.co/t/problem-with-index-folders-in-es-2-3-2/50209/8 "2016-05-17T15:53:47Z")
> [@Jose\_A\_Garcia](#): > > It looks like it should write everything (excluding snapshots) to the same folder, data/elasticsearch/nodes/0/indices/foo. That is a single index called `foo`.[quote="Jose\_A\_Garcia, post:7, topic:50209"] Still, the issue we have shouldn't be related to that, in never tries to write anywhere else than where we expect. The problem comes when we delete the index. After failing to delete the folder the whole cluster breaks down and complains about not being able to obtain the lock. [/quote] You are using ES in a way it was never designed for. I don't know what the exact issue is but I'd suggest you stop this method and just use a disk (or multiple) for everything. ---
**Author:** ![Jose\_A\_Garcia](https://avatars.discourse-cdn.com/v4/letter/j/f19dbf/32.png) [@Jose\_A\_Garcia](https://discuss.elastic.co/u/Jose_A_Garcia)\ **Post date:** [May 17, 2016, 4:01pm UTC](https://discuss.elastic.co/t/problem-with-index-folders-in-es-2-3-2/50209/9 "2016-05-17T16:01:18Z")
> [@warkolm](#): > > That is a single index called foo. That's what I mean, everything for index "foo", goes into that folder. So when I create a mount at "data/elasticsearch/nodes/0/indices/foo" it works. Or it did, now it breaks when it tries to delete the folder. Anyway, I realize that's not a common use case. I'll see if we can work around it. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:50pm UTC](https://discuss.elastic.co/t/problem-with-index-folders-in-es-2-3-2/50209/10 "2017-07-05T22:50:58Z")