# Too Many Open Files **URL:** \ **Category:** Elasticsearch\ **Created:** [January 21, 2014, 3:35pm UTC](https://discuss.elastic.co/t/too-many-open-files/15344 "2014-01-21T15:35:18Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![smonasco\_2](https://avatars.discourse-cdn.com/v4/letter/s/848f3c/32.png) [@smonasco\_2](https://discuss.elastic.co/u/smonasco_2)\ **Post date:** [January 21, 2014, 3:35pm UTC](https://discuss.elastic.co/t/too-many-open-files/15344/1 "2014-01-21T15:35:18Z")
Hi, I am using version 0.19.3 I have the nofile limit set to 128K and am getting errors like [2014-01-18 06:52:54,857][WARN ][netty.channel.socket.nio.NioServerSocketPipelineSink] Failed to initialize an accepted socket. org.elasticsearch.common.netty.channel.ChannelException: Failed to create a selector. at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.start(AbstractNioWorker.java:154) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.register(AbstractNioWorker.java:131) at org.elasticsearch.common.netty.channel.socket.nio.NioServerSocketPipelineSink$Boss.registerAcceptedChannel(NioServerSocketPipelineSink.java:269) at org.elasticsearch.common.netty.channel.socket.nio.NioServerSocketPipelineSink$Boss.run(NioServerSocketPipelineSink.java:231) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Caused by: java.io.IOException: Too many open files at sun.nio.ch.IOUtil.makePipe(Native Method) at sun.nio.ch.EPollSelectorImpl.(EPollSelectorImpl.java:65) at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36) at java.nio.channels.Selector.open(Selector.java:227) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.start(AbstractNioWorker.java:152) ... 7 more I am aware that version 0.19.3 is old. We have been having trouble getting our infrastructure group to build out new nodes so we can have a rolling upgrade with testing for both versions going on. I am now setting the limit to 1048576 as per [http://stackoverflow.com/questions/1212925/on-linux-set-maximum-open-files-to-unlimited-possible](http://stackoverflow.com/questions/1212925/on-linux-set-maximum-open-files-to-unlimited-possible), however, I'm concerned this may cause other issues. If anyone has any suggestions I'd love to hear them. I am using this as fuel for the "please pay attention and get us the support we need so we can upgrade" campaign. --Shannon Monasco -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d76b08e4-d9d2-407e-8443-cb654f381c9a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d76b08e4-d9d2-407e-8443-cb654f381c9a%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![smonasco\_2](https://avatars.discourse-cdn.com/v4/letter/s/848f3c/32.png) [@smonasco\_2](https://discuss.elastic.co/u/smonasco_2)\ **Post date:** [January 21, 2014, 3:42pm UTC](https://discuss.elastic.co/t/too-many-open-files/15344/2 "2014-01-21T15:42:02Z")
Sorry wrong error message. [2014-01-18 06:47:06,232][WARN ][netty.channel.socket.nio.NioServerSocketPipelineSink] Failed to accept a connection. java.io.IOException: Too many open files at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226) at org.elasticsearch.common.netty.channel.socket.nio.NioServerSocketPipelineSink$Boss.run(NioServerSocketPipelineSink.java:227) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) The other posted error message is newer and seems to follow the too many open files error message. --Shannon Monasco On Tuesday, January 21, 2014 8:35:18 AM UTC-7, smonasco wrote: > Hi, > > I am using version 0.19.3 I have the nofile limit set to 128K and am > getting errors like > > [2014-01-18 06:52:54,857][WARN > ][netty.channel.socket.nio.NioServerSocketPipelineSink] Failed to > initialize an accepted socket. > org.elasticsearch.common.netty.channel.ChannelException: Failed to create > a selector. > at > org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.start(AbstractNioWorker.java:154) > at > org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.register(AbstractNioWorker.java:131) > at > org.elasticsearch.common.netty.channel.socket.nio.NioServerSocketPipelineSink$Boss.registerAcceptedChannel(NioServerSocketPipelineSink.java:269) > at > org.elasticsearch.common.netty.channel.socket.nio.NioServerSocketPipelineSink$Boss.run(NioServerSocketPipelineSink.java:231) > at > org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:722) > Caused by: java.io.IOException: Too many open files > at sun.nio.ch.IOUtil.makePipe(Native Method) > at sun.nio.ch.EPollSelectorImpl.(EPollSelectorImpl.java:65) > at > sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36) > at java.nio.channels.Selector.open(Selector.java:227) > at > org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.start(AbstractNioWorker.java:152) > ... 7 more > > I am aware that version 0.19.3 is old. We have been having trouble > getting our infrastructure group to build out new nodes so we can have a > rolling upgrade with testing for both versions going on. I am now setting > the limit to 1048576 as per > [kernel - On Linux - set maximum open files to unlimited. Possible? - Stack Overflow](http://stackoverflow.com/questions/1212925/on-linux-set-maximum-open-files-to-unlimited-possible), > however, I'm concerned this may cause other issues. > > If anyone has any suggestions I'd love to hear them. I am using this as > fuel for the "please pay attention and get us the support we need so we can > upgrade" campaign. > > --Shannon Monasco -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6c77b1f9-7838-40f9-b7e2-6006370265a5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6c77b1f9-7838-40f9-b7e2-6006370265a5%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [January 22, 2014, 5:09pm UTC](https://discuss.elastic.co/t/too-many-open-files/15344/3 "2014-01-22T17:09:42Z")
The first thing to do is check if your limits are actually being persisted and used. The elasticsearch site has a good writeup: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Second, it might be possible that you are reaching the 128k limit. How many shards per node do you have? Do you have non standard merge settings? You can use the status API to find out how many open files you have. I do not have a link since it might have changed since 0.19. Also, be aware that it is not possible to do rolling upgrades with nodes have different major versions of elasticsearch. The underlying data will be fine and does not need to be upgraded, but nodes will not be able to communicate with each other. Cheers, Ivan On Tue, Jan 21, 2014 at 7:42 AM, smonasco [smonasco@gmail.com](mailto:smonasco@gmail.com) wrote: > Sorry wrong error message. > > [2014-01-18 06:47:06,232][WARN > ][netty.channel.socket.nio.NioServerSocketPipelineSink] Failed to accept a > connection. > java.io.IOException: Too many open files > at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) > at > sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226) > at > org.elasticsearch.common.netty.channel.socket.nio.NioServerSocketPipelineSink$Boss.run(NioServerSocketPipelineSink.java:227) > at > org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:722) > > The other posted error message is newer and seems to follow the too many > open files error message. > > --Shannon Monasco > > On Tuesday, January 21, 2014 8:35:18 AM UTC-7, smonasco wrote: > > > Hi, > > > > I am using version 0.19.3 I have the nofile limit set to 128K and am > > getting errors like > > > > [2014-01-18 06:52:54,857][WARN][netty.channel.socket.nio.NioServerSocketPipelineSink] > > Failed to initialize an accepted socket. > > org.elasticsearch.common.netty.channel.ChannelException: Failed to > > create a selector. > > at org.elasticsearch.common.netty.channel.socket.nio. > > AbstractNioWorker.start(AbstractNioWorker.java:154) > > at org.elasticsearch.common.netty.channel.socket.nio. > > AbstractNioWorker.register(AbstractNioWorker.java:131) > > at org.elasticsearch.common.netty.channel.socket.nio. > > NioServerSocketPipelineSink$Boss.registerAcceptedChannel( > > NioServerSocketPipelineSink.java:269) > > at org.elasticsearch.common.netty.channel.socket.nio. > > NioServerSocketPipelineSink$Boss.run(NioServerSocketPipelineSink. > > java:231) > > at org.elasticsearch.common.netty.util.internal. > > DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) > > at java.util.concurrent.ThreadPoolExecutor.runWorker( > > ThreadPoolExecutor.java:1110) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > > ThreadPoolExecutor.java:603) > > at java.lang.Thread.run(Thread.java:722) > > Caused by: java.io.IOException: Too many open files > > at sun.nio.ch.IOUtil.makePipe(Native Method) > > at sun.nio.ch.EPollSelectorImpl.(EPollSelectorImpl.java:65) > > at sun.nio.ch.EPollSelectorProvider.openSelector( > > EPollSelectorProvider.java:36) > > at java.nio.channels.Selector.open(Selector.java:227) > > at org.elasticsearch.common.netty.channel.socket.nio. > > AbstractNioWorker.start(AbstractNioWorker.java:152) > > ... 7 more > > > > I am aware that version 0.19.3 is old. We have been having trouble > > getting our infrastructure group to build out new nodes so we can have a > > rolling upgrade with testing for both versions going on. I am now setting > > the limit to 1048576 as per [http://stackoverflow.com/](http://stackoverflow.com/) > > questions/1212925/on-linux-set-maximum-open-files-to-unlimited-possible, > > however, I'm concerned this may cause other issues. > > > > If anyone has any suggestions I'd love to hear them. I am using this as > > fuel for the "please pay attention and get us the support we need so we can > > upgrade" campaign. > > > > --Shannon Monasco > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/6c77b1f9-7838-40f9-b7e2-6006370265a5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6c77b1f9-7838-40f9-b7e2-6006370265a5%40googlegroups.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDXxLy2Ovm7jMUCREc7Jkva3Hwn3cesu%2BqccGgi%2BP0p\_w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDXxLy2Ovm7jMUCREc7Jkva3Hwn3cesu%2BqccGgi%2BP0p_w%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![smonasco\_2](https://avatars.discourse-cdn.com/v4/letter/s/848f3c/32.png) [@smonasco\_2](https://discuss.elastic.co/u/smonasco_2)\ **Post date:** [March 4, 2014, 7:58pm UTC](https://discuss.elastic.co/t/too-many-open-files/15344/4 "2014-03-04T19:58:08Z")
Sorry to have taken so long to reply. So I went ahead and followed your link. I'd been there before, but decided to give it a deeper look. I found actually, however, that bigdesk told me how many max open files the process was using and from there I was able to determine that my settings in limits.conf was not being honored even though if I switched to the context Elasticsearch was running under I would get the appropriate limits. I then dug into the service script and found someone dropped a ulimit statement into the script that was overwriting the limits.conf setting. Thank you, Shannon Monasco On Wednesday, January 22, 2014 10:09:42 AM UTC-7, Ivan Brusic wrote: > The first thing to do is check if your limits are actually being persisted > and used. The elasticsearch site has a good writeup: > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/tutorials/too-many-open-files/) > > Second, it might be possible that you are reaching the 128k limit. How > many shards per node do you have? Do you have non standard merge settings? > You can use the status API to find out how many open files you have. I do > not have a link since it might have changed since 0.19. > > Also, be aware that it is not possible to do rolling upgrades with nodes > have different major versions of elasticsearch. The underlying data will be > fine and does not need to be upgraded, but nodes will not be able to > communicate with each other. > > Cheers, > > Ivan > > On Tue, Jan 21, 2014 at 7:42 AM, smonasco \<[smon...@gmail.com](mailto:smon...@gmail.com) \ > > > wrote: > > > Sorry wrong error message. > > > > [2014-01-18 06:47:06,232][WARN > > ][netty.channel.socket.nio.NioServerSocketPipelineSink] Failed to accept a > > connection. > > java.io.IOException: Too many open files > > at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) > > at > > sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226) > > at > > org.elasticsearch.common.netty.channel.socket.nio.NioServerSocketPipelineSink$Boss.run(NioServerSocketPipelineSink.java:227) > > at > > org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > at java.lang.Thread.run(Thread.java:722) > > > > The other posted error message is newer and seems to follow the too many > > open files error message. > > > > --Shannon Monasco > > > > On Tuesday, January 21, 2014 8:35:18 AM UTC-7, smonasco wrote: > > > > > Hi, > > > > > > I am using version 0.19.3 I have the nofile limit set to 128K and am > > > getting errors like > > > > > > [2014-01-18 06:52:54,857][WARN][netty.channel.socket.nio.NioServerSocketPipelineSink] > > > Failed to initialize an accepted socket. > > > org.elasticsearch.common.netty.channel.ChannelException: Failed to > > > create a selector. > > > at org.elasticsearch.common.netty.channel.socket.nio. > > > AbstractNioWorker.start(AbstractNioWorker.java:154) > > > at org.elasticsearch.common.netty.channel.socket.nio. > > > AbstractNioWorker.register(AbstractNioWorker.java:131) > > > at org.elasticsearch.common.netty.channel.socket.nio. > > > NioServerSocketPipelineSink$Boss.registerAcceptedChannel( > > > NioServerSocketPipelineSink.java:269) > > > at org.elasticsearch.common.netty.channel.socket.nio. > > > NioServerSocketPipelineSink$Boss.run(NioServerSocketPipelineSink. > > > java:231) > > > at org.elasticsearch.common.netty.util.internal. > > > DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) > > > at java.util.concurrent.ThreadPoolExecutor.runWorker( > > > ThreadPoolExecutor.java:1110) > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > > > ThreadPoolExecutor.java:603) > > > at java.lang.Thread.run(Thread.java:722) > > > Caused by: java.io.IOException: Too many open files > > > at sun.nio.ch.IOUtil.makePipe(Native Method) > > > at sun.nio.ch.EPollSelectorImpl.(EPollSelectorImpl.java: > > > 65) > > > at sun.nio.ch.EPollSelectorProvider.openSelector( > > > EPollSelectorProvider.java:36) > > > at java.nio.channels.Selector.open(Selector.java:227) > > > at org.elasticsearch.common.netty.channel.socket.nio. > > > AbstractNioWorker.start(AbstractNioWorker.java:152) > > > ... 7 more > > > > > > I am aware that version 0.19.3 is old. We have been having trouble > > > getting our infrastructure group to build out new nodes so we can have a > > > rolling upgrade with testing for both versions going on. I am now setting > > > the limit to 1048576 as per [http://stackoverflow.com/](http://stackoverflow.com/) > > > questions/1212925/on-linux-set-maximum-open-files-to-unlimited-possible, > > > however, I'm concerned this may cause other issues. > > > > > > If anyone has any suggestions I'd love to hear them. I am using this as > > > fuel for the "please pay attention and get us the support we need so we can > > > upgrade" campaign. > > > > > > --Shannon Monasco > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/6c77b1f9-7838-40f9-b7e2-6006370265a5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6c77b1f9-7838-40f9-b7e2-6006370265a5%40googlegroups.com) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5da7c5f2-90ce-46ce-abc8-36900fb6b006%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5da7c5f2-90ce-46ce-abc8-36900fb6b006%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:45am UTC](https://discuss.elastic.co/t/too-many-open-files/15344/5 "2017-07-06T01:45:46Z")