# ElasticSearch fails on NFS, makes tons of empty directories in nodes **URL:** \ **Category:** Elasticsearch\ **Created:** [February 22, 2012, 12:15am UTC](https://discuss.elastic.co/t/elasticsearch-fails-on-nfs-makes-tons-of-empty-directories-in-nodes/6775 "2012-02-22T00:15:51Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Gregory\_Rice](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gregory_rice/32/2884_2.png) [@Gregory\_Rice](https://discuss.elastic.co/u/Gregory_Rice)\ **Post date:** [February 22, 2012, 12:15am UTC](https://discuss.elastic.co/t/elasticsearch-fails-on-nfs-makes-tons-of-empty-directories-in-nodes/6775/1 "2012-02-22T00:15:51Z")
Hey guys, I've changed data.path to an NFS share mounted on my single-node elasticsearch machine. The monolithic elasticsearch has no problem creating the elasticsearch directory, or the nodes directory, but while it has no trouble making a node directory and filling it with data on local drives, on my NFS share it simply creates a bunch of directories in the /nfs\_share/data/elasticsearch/nodes directory at a rate about one every 30 seconds until after a while, there's like fifty nodes listed in the directory, each completely empty except for a node.lock file. Obviously it's able to create the node.lock file and the directory structure, so it's not a permissions issue, but is there a particular reason something like this would happen on NFS when there's no problem getting data onto a local disk, with only one directory created under "nodes"? Anyone seen anything like this or know what I can do to help figure out what's going on? Thanks, Greg ---
**Author:** ![Radu\_Gheorghe1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radu_gheorghe1/32/2688_2.png) [@Radu\_Gheorghe1](https://discuss.elastic.co/u/Radu_Gheorghe1)\ **Post date:** [February 22, 2012, 6:55am UTC](https://discuss.elastic.co/t/elasticsearch-fails-on-nfs-makes-tons-of-empty-directories-in-nodes/6775/2 "2012-02-22T06:55:31Z")
Hi Greg, In Elasticsearch, do you have the Gateway left to the default "local", or did you change it to "fs"? We're having two nodes here that write on an NFS share with the Gateway set to "local", and we had a lot of strange permission issues. What works now is that we have all\_squash set and the data directory belongs to nobody:nogroup. Hope this helps, Radu ---
**Author:** ![Gregory\_Rice](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gregory_rice/32/2884_2.png) [@Gregory\_Rice](https://discuss.elastic.co/u/Gregory_Rice)\ **Post date:** [February 22, 2012, 8:23pm UTC](https://discuss.elastic.co/t/elasticsearch-fails-on-nfs-makes-tons-of-empty-directories-in-nodes/6775/3 "2012-02-22T20:23:46Z")
Radu, I tried changing the gateway to "fs". I tried moving the gateway location to the share. What happens is, when I set the data.path directory to a directory on the fileshare, elasticsearch doesn't even really start up. It gets as far as: [2012-02-22 20:06:12,212][INFO][node] [Ziggy Pig] {0.18.7}[3907]: initializing ... [2012-02-22 20:06:12,218][INFO][plugins] [Ziggy Pig] loaded , sites And hangs. Changing the gateway type or gateway location did nothing. Remember, it doesn't look like permissions, as Elasticsearch has no problem creating node/0 node/1 node/2 node/3 ENDLESSLY on the NFS drive, so obviously it's able to write. It simply cannot initiate the node, so it tries creating another one. Any other ideas? It's not able to create anything but endless new nodes (as it never is able to initialize one successfully, I guess) and then puts a node.lock file in them and moves on and makes another. Thanks, Greg Rice On Feb 21, 10:55 pm, Radu Gheorghe [radu0gheor...@gmail.com](mailto:radu0gheor...@gmail.com) wrote: > Hi Greg, > > In Elasticsearch, do you have the Gateway left to the default "local", > or did you change it to "fs"? > > We're having two nodes here that write on an NFS share with the > Gateway set to "local", and we had a lot of strange permission issues. > What works now is that we have all\_squash set and the data directory > belongs to nobody:nogroup. > > Hope this helps, > Radu ---
**Author:** ![Radu\_Gheorghe1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radu_gheorghe1/32/2688_2.png) [@Radu\_Gheorghe1](https://discuss.elastic.co/u/Radu_Gheorghe1)\ **Post date:** [February 23, 2012, 7:13am UTC](https://discuss.elastic.co/t/elasticsearch-fails-on-nfs-makes-tons-of-empty-directories-in-nodes/6775/4 "2012-02-23T07:13:45Z")
Nope, I don't have any other ideas. I would just try with all\_squash and chown -R nobody:nogroup on the data directory. And with the default "local" gateway. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 26, 2012, 9:57am UTC](https://discuss.elastic.co/t/elasticsearch-fails-on-nfs-makes-tons-of-empty-directories-in-nodes/6775/5 "2012-02-26T09:57:05Z")
It sounds like it failed to acquire an exclusive lock on the node.lock file (which is needs to do). We can try and see why the problem happens, but why use NFS to store the data? On Wednesday, February 22, 2012 at 2:15 AM, Gregory Rice wrote: > Hey guys, > > I've changed data.path to an NFS share mounted on my single-node > elasticsearch machine. The monolithic elasticsearch has no problem > creating the > elasticsearch directory, or the nodes directory, but while it has no > trouble making a node directory and filling it with data on local > drives, > on my NFS share it simply creates a bunch of directories in the > /nfs\_share/data/elasticsearch/nodes directory at a rate about one > every 30 > seconds until after a while, there's like fifty nodes listed in the > directory, each completely empty except for a node.lock file. > > Obviously it's able to create the node.lock file and the directory > structure, so it's not a permissions issue, but is there a particular > reason something like this would happen on NFS when there's no problem > getting data onto a local disk, with only one directory created under > "nodes"? > > Anyone seen anything like this or know what I can do to help figure > out > what's going on? > > Thanks, > Greg ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:38am UTC](https://discuss.elastic.co/t/elasticsearch-fails-on-nfs-makes-tons-of-empty-directories-in-nodes/6775/6 "2017-07-06T03:38:15Z")