# Too many open files

**URL:** <https://discuss.elastic.co/t/too-many-open-files/18729>\
**Category:** Elasticsearch\
**Created:** [July 18, 2014, 4:40am UTC](https://discuss.elastic.co/t/too-many-open-files/18729 "2014-07-18T04:40:41Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Seungjin\_Lee](https://avatars.discourse-cdn.com/v4/letter/s/2acd7d/32.png) [@Seungjin\_Lee](https://discuss.elastic.co/u/Seungjin_Lee)\
**Post date:** [July 18, 2014, 4:40am UTC](https://discuss.elastic.co/t/too-many-open-files/18729/1 "2014-07-18T04:40:41Z")

</div>

hello, I'm using elasticsearch with storm, Java TransportClient.

I have total 128 threads across machines which communicate with  
elasticsearch cluster.

From time to time, error below occurs

org.elasticsearch.common.netty.channel.ChannelException: Failed to create a  
selector.  
at  
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:343)  
at  
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.(AbstractNioSelector.java:100)  
at  
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.(AbstractNioWorker.java:52)  
at  
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.(NioWorker.java:45)  
at  
org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:45)  
at  
org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:28)  
at  
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorkerPool.newWorker(AbstractNioWorkerPool.java:143)  
at  
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:81)  
at  
org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.(NioWorkerPool.java:39)  
at  
org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.(NioWorkerPool.java:33)  
at  
org.elasticsearch.transport.netty.NettyTransport.doStart(NettyTransport.java:254)  
at  
org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)  
at  
org.elasticsearch.transport.TransportService.doStart(TransportService.java:92)  
at  
org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)  
at  
org.elasticsearch.client.transport.TransportClient.(TransportClient.java:189)  
at  
org.elasticsearch.client.transport.TransportClient.(TransportClient.java:125)  
at  
com.naver.labs.nelo2.notifier.utils.ElasticSearchUtil.prepareElasticSearch(ElasticSearchUtil.java:30)  
at  
com.naver.labs.nelo2.notifier.bolt.PercolatorBolt.prepare(PercolatorBolt.java:48)  
at  
backtype.storm.topology.BasicBoltExecutor.prepare(BasicBoltExecutor.java:43)  
at backtype.storm.daemon.executor$fn\_\_5641$fn\_\_5653.invoke(executor.clj:690)  
at backtype.storm.util$async\_loop$fn\_\_457.invoke(util.clj:429)  
at clojure.lang.AFn.run(AFn.java:24)  
at java.lang.Thread.run(Thread.java:745)  
Caused by: java.io.IOException: too many open files  
at sun.nio.ch.IOUtil.makePipe(Native Method)  
at sun.nio.ch.EPollSelectorImpl.(EPollSelectorImpl.java:65)  
at  
sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)  
at java.nio.channels.Selector.open(Selector.java:227)  
at  
org.elasticsearch.common.netty.channel.socket.nio.SelectorUtil.open(SelectorUtil.java:63)  
at  
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:341)  
... 22 more

code itself is very simple

I get client as follows

```
    Settings settings =
            ImmutableSettings.settingsBuilder().put("cluster.name",

```

clusterName).put("client.transport.sniff", "true").build();  
List transportAddressList = new  
ArrayList();  
for (String host : ESHost) {  
transportAddressList.add(new InetSocketTransportAddress(host,  
ESPort));  
}  
return new TransportClient(settings)  
.addTransportAddresses(transportAddressList.toArray(new  
InetSocketTransportAddress[transportAddressList.size()]));

and for each execution, it percolates as follows

```
    return

```

client.preparePercolate().setIndices(indexName).setDocumentType(projectName).setPercolateDoc(docBuilder().setDoc(log)).setRouting(projectName).setPercolateFilter(FilterBuilders.termFilter("projects",  
projectName).cache(true)).execute().actionGet();

ES cluster consists of 5 machines with almost default setting.

What can be the cause of this problem?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAL3\_U43SQ2KCGFCQ%3DcisaLGQXxdg\_r1mzjrcunreaJOj0Ln-%2BQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAL3_U43SQ2KCGFCQ%3DcisaLGQXxdg_r1mzjrcunreaJOj0Ln-%2BQ%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![andrew](https://avatars.discourse-cdn.com/v4/letter/a/f19dbf/32.png) [@andrew](https://discuss.elastic.co/u/andrew)\
**Post date:** [July 18, 2014, 4:46am UTC](https://discuss.elastic.co/t/too-many-open-files/18729/2 "2014-07-18T04:46:42Z")

</div>

This is a fairly common problem and not necessarily specific to Elasticsearch. It is simple to solve. In Linux you can increase the operating system's max file descriptor limit. Other Unix-like operating systems have the same concept. You can find how to do this for your specific Linux distribution from a little googling on "linux max file descriptor".

Cheers.

On Jul 17, 2014, at 9:40 PM, Seungjin Lee [sweetest0409@gmail.com](mailto:sweetest0409@gmail.com) wrote:

> hello, I'm using elasticsearch with storm, Java TransportClient.
> 
> I have total 128 threads across machines which communicate with elasticsearch cluster.
> 
> From time to time, error below occurs
> 
> org.elasticsearch.common.netty.channel.ChannelException: Failed to create a selector.  
> at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:343)  
> at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.(AbstractNioSelector.java:100)  
> at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.(AbstractNioWorker.java:52)  
> at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.(NioWorker.java:45)  
> at org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:45)  
> at org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:28)  
> at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorkerPool.newWorker(AbstractNioWorkerPool.java:143)  
> at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:81)  
> at org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.(NioWorkerPool.java:39)  
> at org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.(NioWorkerPool.java:33)  
> at org.elasticsearch.transport.netty.NettyTransport.doStart(NettyTransport.java:254)  
> at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)  
> at org.elasticsearch.transport.TransportService.doStart(TransportService.java:92)  
> at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)  
> at org.elasticsearch.client.transport.TransportClient.(TransportClient.java:189)  
> at org.elasticsearch.client.transport.TransportClient.(TransportClient.java:125)  
> at com.naver.labs.nelo2.notifier.utils.ElasticSearchUtil.prepareElasticSearch(ElasticSearchUtil.java:30)  
> at com.naver.labs.nelo2.notifier.bolt.PercolatorBolt.prepare(PercolatorBolt.java:48)  
> at backtype.storm.topology.BasicBoltExecutor.prepare(BasicBoltExecutor.java:43)  
> at backtype.storm.daemon.executor$fn\_\_5641$fn\_\_5653.invoke(executor.clj:690)  
> at backtype.storm.util$async\_loop$fn\_\_457.invoke(util.clj:429)  
> at clojure.lang.AFn.run(AFn.java:24)  
> at java.lang.Thread.run(Thread.java:745)  
> Caused by: java.io.IOException: too many open files  
> at sun.nio.ch.IOUtil.makePipe(Native Method)  
> at sun.nio.ch.EPollSelectorImpl.(EPollSelectorImpl.java:65)  
> at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)  
> at java.nio.channels.Selector.open(Selector.java:227)  
> at org.elasticsearch.common.netty.channel.socket.nio.SelectorUtil.open(SelectorUtil.java:63)  
> at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:341)  
> ... 22 more
> 
> code itself is very simple
> 
> I get client as follows
> 
> ```
> Settings settings =
> ImmutableSettings.settingsBuilder().put("cluster.name", clusterName).put("client.transport.sniff", "true").build();
> List<InetSocketTransportAddress> transportAddressList = new ArrayList<InetSocketTransportAddress>();
> for (String host : ESHost) {
> transportAddressList.add(new InetSocketTransportAddress(host, ESPort));
> }
> return new TransportClient(settings)
> .addTransportAddresses(transportAddressList.toArray(new InetSocketTransportAddress[transportAddressList.size()]));
> 
> ```
> 
> and for each execution, it percolates as follows
> 
> ```
> return client.preparePercolate().setIndices(indexName).setDocumentType(projectName).setPercolateDoc(docBuilder().setDoc(log)).setRouting(projectName).setPercolateFilter(FilterBuilders.termFilter("projects", projectName).cache(true)).execute().actionGet();
> 
> ```
> 
> ES cluster consists of 5 machines with almost default setting.
> 
> What can be the cause of this problem?
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAL3\_U43SQ2KCGFCQ%3DcisaLGQXxdg\_r1mzjrcunreaJOj0Ln-%2BQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAL3_U43SQ2KCGFCQ%3DcisaLGQXxdg_r1mzjrcunreaJOj0Ln-%2BQ%40mail.gmail.com).  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/DDFC2392-63BE-49DA-BDE8-48F25185B444%40elasticsearch.com](https://groups.google.com/d/msgid/elasticsearch/DDFC2392-63BE-49DA-BDE8-48F25185B444%40elasticsearch.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:15am UTC](https://discuss.elastic.co/t/too-many-open-files/18729/3 "2017-07-06T01:15:04Z")

</div>