# Excessive Number of Open File Descriptor Count **URL:** \ **Category:** Elasticsearch\ **Created:** [April 16, 2012, 6:16am UTC](https://discuss.elastic.co/t/excessive-number-of-open-file-descriptor-count/7341 "2012-04-16T06:16:25Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Stanislas\_Polu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stanislas_polu/32/2873_2.png) [@Stanislas\_Polu](https://discuss.elastic.co/u/Stanislas_Polu)\ **Post date:** [April 16, 2012, 6:16am UTC](https://discuss.elastic.co/t/excessive-number-of-open-file-descriptor-count/7341/1 "2012-04-16T06:16:25Z")
Hi everyone, I run a cluster of 3 instances with local storage. I'm indexing ~60 elements per seconds and have been doing so for the past month. The number of concurrent queries is fairly low at the moment. The number of file descriptors needed has been constantly growing and has now grown to a point where it feels like a real issue my current limit is 946000 and it's still not enough Is there anything I should do? Should I attempt to grant unlimited number of file descriptors? Thanks for your help! Best, -stan -- Stanislas Polu Mo: +33 6 83 71 90 04 | Tw: @spolu | [http://teleportd.com](http://teleportd.com) | Realtime Photo Search ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 16, 2012, 7:13am UTC](https://discuss.elastic.co/t/excessive-number-of-open-file-descriptor-count/7341/2 "2012-04-16T07:13:18Z")
Does your number of shards (indices) grows? Or you just continue to index into the same index more and more data? On Mon, Apr 16, 2012 at 9:16 AM, Stanislas Polu [polu.stanislas@gmail.com](mailto:polu.stanislas@gmail.com)wrote: > Hi everyone, > > I run a cluster of 3 instances with local storage. > I'm indexing ~60 elements per seconds and have been doing so for the past > month. > The number of concurrent queries is fairly low at the moment. > > The number of file descriptors needed has been constantly growing and has > now grown to a point where it feels like a real issue > my current limit is 946000 and it's still not enough > > Is there anything I should do? > Should I attempt to grant unlimited number of file descriptors? > > Thanks for your help! > > Best, > > -stan > > -- > Stanislas Polu > Mo: +33 6 83 71 90 04 | Tw: @spolu | [http://teleportd.com](http://teleportd.com) | Realtime > Photo Search ---
**Author:** ![Stanislas\_Polu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stanislas_polu/32/2873_2.png) [@Stanislas\_Polu](https://discuss.elastic.co/u/Stanislas_Polu)\ **Post date:** [April 16, 2012, 7:21am UTC](https://discuss.elastic.co/t/excessive-number-of-open-file-descriptor-count/7341/3 "2012-04-16T07:21:35Z")
Nope posting to the same index. On Monday, April 16, 2012 9:13:18 AM UTC+2, kimchy wrote: > Does your number of shards (indices) grows? Or you just continue to index > into the same index more and more data? > > On Mon, Apr 16, 2012 at 9:16 AM, Stanislas Polu [polu.stanislas@gmail.com](mailto:polu.stanislas@gmail.com)wrote: > > > Hi everyone, > > > > I run a cluster of 3 instances with local storage. > > I'm indexing ~60 elements per seconds and have been doing so for the past > > month. > > The number of concurrent queries is fairly low at the moment. > > > > The number of file descriptors needed has been constantly growing and has > > now grown to a point where it feels like a real issue > > my current limit is 946000 and it's still not enough > > > > Is there anything I should do? > > Should I attempt to grant unlimited number of file descriptors? > > > > Thanks for your help! > > > > Best, > > > > -stan > > > > -- > > Stanislas Polu > > Mo: +33 6 83 71 90 04 | Tw: @spolu | [http://teleportd.com](http://teleportd.com) | Realtime > > Photo Search ---
**Author:** ![Stanislas\_Polu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stanislas_polu/32/2873_2.png) [@Stanislas\_Polu](https://discuss.elastic.co/u/Stanislas_Polu)\ **Post date:** [April 16, 2012, 7:33am UTC](https://discuss.elastic.co/t/excessive-number-of-open-file-descriptor-count/7341/4 "2012-04-16T07:33:08Z")
I've restarted everything and I'm now at 1106 open file descriptors (lsof -p | wc -l) On Monday, April 16, 2012 9:21:35 AM UTC+2, Stanislas Polu wrote: > Nope posting to the same index. > > On Monday, April 16, 2012 9:13:18 AM UTC+2, kimchy wrote: > > > Does your number of shards (indices) grows? Or you just continue to index > > into the same index more and more data? > > > > On Mon, Apr 16, 2012 at 9:16 AM, Stanislas Polu \<[polu.stanislas@gmail.com](mailto:polu.stanislas@gmail.com) > > > > > wrote: > > > > > Hi everyone, > > > > > > I run a cluster of 3 instances with local storage. > > > I'm indexing ~60 elements per seconds and have been doing so for the > > > past month. > > > The number of concurrent queries is fairly low at the moment. > > > > > > The number of file descriptors needed has been constantly growing and > > > has now grown to a point where it feels like a real issue > > > my current limit is 946000 and it's still not enough > > > > > > Is there anything I should do? > > > Should I attempt to grant unlimited number of file descriptors? > > > > > > Thanks for your help! > > > > > > Best, > > > > > > -stan > > > > > > -- > > > Stanislas Polu > > > Mo: +33 6 83 71 90 04 | Tw: @spolu | [http://teleportd.com](http://teleportd.com) | Realtime > > > Photo Search ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 17, 2012, 8:25am UTC](https://discuss.elastic.co/t/excessive-number-of-open-file-descriptor-count/7341/5 "2012-04-17T08:25:55Z")
Next time it happens, can you double check which file descriptors are being used (and gist the output of it)? What I mean, can you double check that the open file descriptors are mainly coming from the data location ES stores the files? It might be open sockets for example. On Mon, Apr 16, 2012 at 10:33 AM, Stanislas Polu [polu.stanislas@gmail.com](mailto:polu.stanislas@gmail.com)wrote: > I've restarted everything and I'm now at 1106 open file descriptors (lsof > -p | wc -l) > > On Monday, April 16, 2012 9:21:35 AM UTC+2, Stanislas Polu wrote: > > > Nope posting to the same index. > > > > On Monday, April 16, 2012 9:13:18 AM UTC+2, kimchy wrote: > > > > > Does your number of shards (indices) grows? Or you just continue to > > > index into the same index more and more data? > > > > > > On Mon, Apr 16, 2012 at 9:16 AM, Stanislas Polu \< > > > [polu.stanislas@gmail.com](mailto:polu.stanislas@gmail.com)\> wrote: > > > > > > > Hi everyone, > > > > > > > > I run a cluster of 3 instances with local storage. > > > > I'm indexing ~60 elements per seconds and have been doing so for the > > > > past month. > > > > The number of concurrent queries is fairly low at the moment. > > > > > > > > The number of file descriptors needed has been constantly growing and > > > > has now grown to a point where it feels like a real issue > > > > my current limit is 946000 and it's still not enough > > > > > > > > Is there anything I should do? > > > > Should I attempt to grant unlimited number of file descriptors? > > > > > > > > Thanks for your help! > > > > > > > > Best, > > > > > > > > -stan > > > > > > > > -- > > > > Stanislas Polu > > > > Mo: +33 6 83 71 90 04 | Tw: @spolu | [http://teleportd.com](http://teleportd.com) | \*\*Realtime > > > > Photo Search ---
**Author:** ![vibin](https://avatars.discourse-cdn.com/v4/letter/v/2bfe46/32.png) [@vibin](https://discuss.elastic.co/u/vibin)\ **Post date:** [April 27, 2012, 2:50pm UTC](https://discuss.elastic.co/t/excessive-number-of-open-file-descriptor-count/7341/6 "2012-04-27T14:50:43Z")
Hi Plou/Shay, I am having the same problem in the Elastic Search 0.19.1 version. Did you do anything to fix it? I have another cluster doing the same thing but the number of files are not as much as like this. The version I am running there with 0.18.5. I am not sure if this a version problem. but his seems big. Let me know any finds. Thank you! -vibin On Tue, Apr 17, 2012 at 1:55 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > Next time it happens, can you double check which file descriptors are being > used (and gist the output of it)? What I mean, can you double check that the > open file descriptors are mainly coming from the data location ES stores the > files? It might be open sockets for example. > > On Mon, Apr 16, 2012 at 10:33 AM, Stanislas Polu [polu.stanislas@gmail.com](mailto:polu.stanislas@gmail.com) > wrote: > > > I've restarted everything and I'm now at 1106 open file descriptors (lsof > > -p | wc -l) > > > > On Monday, April 16, 2012 9:21:35 AM UTC+2, Stanislas Polu wrote: > > > > > Nope posting to the same index. > > > > > > On Monday, April 16, 2012 9:13:18 AM UTC+2, kimchy wrote: > > > > > > > Does your number of shards (indices) grows? Or you just continue to > > > > index into the same index more and more data? > > > > > > > > On Mon, Apr 16, 2012 at 9:16 AM, Stanislas Polu > > > > [polu.stanislas@gmail.com](mailto:polu.stanislas@gmail.com) wrote: > > > > > > > > > Hi everyone, > > > > > > > > > > I run a cluster of 3 instances with local storage. > > > > > I'm indexing ~60 elements per seconds and have been doing so for the > > > > > past month. > > > > > The number of concurrent queries is fairly low at the moment. > > > > > > > > > > The number of file descriptors needed has been constantly growing and > > > > > has now grown to a point where it feels like a real issue > > > > > my current limit is 946000 and it's still not enough > > > > > > > > > > Is there anything I should do? > > > > > Should I attempt to grant unlimited number of file descriptors? > > > > > > > > > > Thanks for your help! > > > > > > > > > > Best, > > > > > > > > > > -stan > > > > > > > > > > -- > > > > > Stanislas Polu > > > > > Mo: +33 6 83 71 90 04 | Tw: @spolu | [http://teleportd.com](http://teleportd.com) | Realtime > > > > > Photo Search -- Regards, - vibindhas ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 29, 2012, 3:47pm UTC](https://discuss.elastic.co/t/excessive-number-of-open-file-descriptor-count/7341/7 "2012-04-29T15:47:58Z")
Can you gist the list of open file handles you have? The result of lsof -p basically. On Fri, Apr 27, 2012 at 5:50 PM, vibin dhas [vibindhas@gmail.com](mailto:vibindhas@gmail.com) wrote: > Hi Plou/Shay, > > I am having the same problem in the Elastic Search 0.19.1 version. Did > you do anything to fix it? I have another cluster doing the same thing > but the number of files are not as much as like this. The version I am > running there with 0.18.5. I am not sure if this a version problem. > but his seems big. > > Let me know any finds. > > Thank you! > > -vibin > > On Tue, Apr 17, 2012 at 1:55 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > > Next time it happens, can you double check which file descriptors are > > being > > used (and gist the output of it)? What I mean, can you double check that > > the > > open file descriptors are mainly coming from the data location ES stores > > the > > files? It might be open sockets for example. > > > > On Mon, Apr 16, 2012 at 10:33 AM, Stanislas Polu \< > > [polu.stanislas@gmail.com](mailto:polu.stanislas@gmail.com)\> > > wrote: > > > > > I've restarted everything and I'm now at 1106 open file descriptors > > > (lsof > > > -p | wc -l) > > > > > > On Monday, April 16, 2012 9:21:35 AM UTC+2, Stanislas Polu wrote: > > > > > > > Nope posting to the same index. > > > > > > > > On Monday, April 16, 2012 9:13:18 AM UTC+2, kimchy wrote: > > > > > > > > > Does your number of shards (indices) grows? Or you just continue to > > > > > index into the same index more and more data? > > > > > > > > > > On Mon, Apr 16, 2012 at 9:16 AM, Stanislas Polu > > > > > [polu.stanislas@gmail.com](mailto:polu.stanislas@gmail.com) wrote: > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > I run a cluster of 3 instances with local storage. > > > > > > I'm indexing ~60 elements per seconds and have been doing so for the > > > > > > past month. > > > > > > The number of concurrent queries is fairly low at the moment. > > > > > > > > > > > > The number of file descriptors needed has been constantly growing and > > > > > > has now grown to a point where it feels like a real issue > > > > > > my current limit is 946000 and it's still not enough > > > > > > > > > > > > Is there anything I should do? > > > > > > Should I attempt to grant unlimited number of file descriptors? > > > > > > > > > > > > Thanks for your help! > > > > > > > > > > > > Best, > > > > > > > > > > > > -stan > > > > > > > > > > > > -- > > > > > > Stanislas Polu > > > > > > Mo: +33 6 83 71 90 04 | Tw: @spolu | [http://teleportd.com](http://teleportd.com) | Realtime > > > > > > Photo Search > > -- > Regards, > > - vibindhas ---
**Author:** ![Stanislas\_Polu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stanislas_polu/32/2873_2.png) [@Stanislas\_Polu](https://discuss.elastic.co/u/Stanislas_Polu)\ **Post date:** [May 15, 2012, 9:12am UTC](https://discuss.elastic.co/t/excessive-number-of-open-file-descriptor-count/7341/8 "2012-05-15T09:12:51Z")
Sorry for the latency here. I think it was mainly sockets indeed. I haven't exeperienced the pb anymore since I limited the number of concurrent open connections to my ES instance with an external queue mechanism. Best, -stan -- Stanislas Polu Mo: +33 6 83 71 90 04 | Tw: @spolu | [http://teleportd.com](http://teleportd.com) | Realtime Photo Search On Tue, Apr 17, 2012 at 10:25 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > Next time it happens, can you double check which file descriptors are > being used (and gist the output of it)? What I mean, can you double check > that the open file descriptors are mainly coming from the data location ES > stores the files? It might be open sockets for example. > > On Mon, Apr 16, 2012 at 10:33 AM, Stanislas Polu \<[polu.stanislas@gmail.com](mailto:polu.stanislas@gmail.com) > > > wrote: > > > I've restarted everything and I'm now at 1106 open file descriptors (lsof > > -p | wc -l) > > > > On Monday, April 16, 2012 9:21:35 AM UTC+2, Stanislas Polu wrote: > > > > > Nope posting to the same index. > > > > > > On Monday, April 16, 2012 9:13:18 AM UTC+2, kimchy wrote: > > > > > > > Does your number of shards (indices) grows? Or you just continue to > > > > index into the same index more and more data? > > > > > > > > On Mon, Apr 16, 2012 at 9:16 AM, Stanislas Polu \< > > > > [polu.stanislas@gmail.com](mailto:polu.stanislas@gmail.com)\> wrote: > > > > > > > > > Hi everyone, > > > > > > > > > > I run a cluster of 3 instances with local storage. > > > > > I'm indexing ~60 elements per seconds and have been doing so for the > > > > > past month. > > > > > The number of concurrent queries is fairly low at the moment. > > > > > > > > > > The number of file descriptors needed has been constantly growing and > > > > > has now grown to a point where it feels like a real issue > > > > > my current limit is 946000 and it's still not enough > > > > > > > > > > Is there anything I should do? > > > > > Should I attempt to grant unlimited number of file descriptors? > > > > > > > > > > Thanks for your help! > > > > > > > > > > Best, > > > > > > > > > > -stan > > > > > > > > > > -- > > > > > Stanislas Polu > > > > > Mo: +33 6 83 71 90 04 | Tw: @spolu | [http://teleportd.com](http://teleportd.com) | \*\*Realtime > > > > > Photo Search ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:28am UTC](https://discuss.elastic.co/t/excessive-number-of-open-file-descriptor-count/7341/9 "2017-07-06T03:28:52Z")