# Too many open files warning **URL:** \ **Category:** Elasticsearch\ **Created:** [April 9, 2013, 12:02pm UTC](https://discuss.elastic.co/t/too-many-open-files-warning/11503 "2013-04-09T12:02:19Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![ovamsikrishna](https://avatars.discourse-cdn.com/v4/letter/o/e8c25b/32.png) [@ovamsikrishna](https://discuss.elastic.co/u/ovamsikrishna)\ **Post date:** [April 9, 2013, 12:02pm UTC](https://discuss.elastic.co/t/too-many-open-files-warning/11503/1 "2013-04-09T12:02:19Z")
Getting the below warning messages continuously. Not sure what should be done. Saw some of the relevant posts asking to increase the number of file descriptors. How to do the same? Even if I increase now, Will I encounter the same issue on addition of new indices. (presently working with around 400 indices, 6 shards and 1 replica). The number of indices tend to grow more. [03:58:24,165][WARN][cluster.action.shard] [node1] received shard failed for [index9][2], node[node\_hash3], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[index9][2] failed recovery]; nested: EngineCreationFailureException[[index9][2] failed to open reader on writer]; nested: FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index9/2/index/segments\_1 (Too many open files)]; ]] [03:58:24,166][WARN][cluster.action.shard] [node1] received shard failed for [index15][0], node[node\_hash2], [P], s[INITIALIZING], reason [Failed to create shard, message [IndexShardCreationException[[index15][0] failed to create shard]; nested: IOException[directory '/data/elasticsearch/whatever/nodes/0/indices/index15/0/index' exists and is a directory, but cannot be listed: list() returned null]; ]] [03:58:24,195][WARN][cluster.action.shard] [node1] received shard failed for [index16][3], node[node\_hash3], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[index16][3] failed recovery]; nested: EngineCreationFailureException[[index16][3] failed to open reader on writer]; nested: FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index16/3/index/segments\_1 (Too many open files)]; ]] [03:58:24,196][WARN][cluster.action.shard] [node1] received shard failed for [index17][0], node[node\_hash3], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[index17][0] failed recovery]; nested: EngineCreationFailureException[[index17][0] failed to open reader on writer]; nested: FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index17/0/index/segments\_1 (Too many open files)]; ]] [03:58:24,198][WARN][cluster.action.shard] [node1] received shard failed for [index21][4], node[node\_hash3], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[index21][4] failed recovery]; nested: EngineCreationFailureException[[index21][4] failed to create engine]; nested: LockReleaseFailedException[Cannot forcefully unlock a NativeFSLock which is held by another indexer component: /data/elasticsearch/whatever/nodes/0/indices/index21/4/index/write.lock]; ]] Output of nodes api curl -XGET '[http://localhost:9200/\_nodes?os=true&process=true&pretty=true](http://localhost:9200/_nodes?os=true&process=true&pretty=true)' { "ok" : true, "cluster\_name" : "whatever", "nodes" : { "node\_hash1" : { "name" : "node1", "transport\_address" : "transportip1", "hostname" : "myhostip1", "version" : "0.20.4", "http\_address" : "httpip1", "attributes" : { "data" : "false", "master" : "true" }, "os" : { "refresh\_interval" : 1000, "available\_processors" : 8, "cpu" : { "vendor" : "Intel", "model" : "Xeon", "mhz" : 2133, "total\_cores" : 8, "total\_sockets" : 8, "cores\_per\_socket" : 16, "cache\_size" : "4kb", "cache\_size\_in\_bytes" : 4096 }, "mem" : { "total" : "7gb", "total\_in\_bytes" : 7516336128 }, "swap" : { "total" : "30gb", "total\_in\_bytes" : 32218378240 } }, "process" : { "refresh\_interval" : 1000, "id" : 26188, "max\_file\_descriptors" : 16384 } }, "node\_hash2" : { "name" : "node2", "transport\_address" : "transportip2", "hostname" : "myhostip2", "version" : "0.20.4", "attributes" : { "master" : "false" }, "os" : { "refresh\_interval" : 1000, "available\_processors" : 4, "cpu" : { "vendor" : "Intel", "model" : "Xeon", "mhz" : 2400, "total\_cores" : 4, "total\_sockets" : 4, "cores\_per\_socket" : 32, "cache\_size" : "20kb", "cache\_size\_in\_bytes" : 20480 }, "mem" : { "total" : "34.1gb", "total\_in\_bytes" : 36700303360 }, "swap" : { "total" : "0b", "total\_in\_bytes" : 0 } }, "process" : { "refresh\_interval" : 1000, "id" : 24883, "max\_file\_descriptors" : 16384 } }, "node\_hash3" : { "name" : "node3", "transport\_address" : "transportip3", "hostname" : "myhostip3", "version" : "0.20.4", "attributes" : { "master" : "false" }, "os" : { "refresh\_interval" : 1000, "available\_processors" : 4, "cpu" : { "vendor" : "Intel", "model" : "Xeon", "mhz" : 2666, "total\_cores" : 4, "total\_sockets" : 4, "cores\_per\_socket" : 16, "cache\_size" : "8kb", "cache\_size\_in\_bytes" : 8192 }, "mem" : { "total" : "34.1gb", "total\_in\_bytes" : 36700303360 }, "swap" : { "total" : "0b", "total\_in\_bytes" : 0 } }, "process" : { "refresh\_interval" : 1000, "id" : 25328, "max\_file\_descriptors" : 16384 } } } ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [April 10, 2013, 5:19pm UTC](https://discuss.elastic.co/t/too-many-open-files-warning/11503/2 "2013-04-10T17:19:41Z")
Hello, Note this part: ``` "process" : { "refresh_interval" : 1000, "id" : 25328, "max_file_descriptors" : *16384* } ``` Don't be afraid to increase quadruple this. Your 600 indices each have some number of shards, each shard some number of replicas, each of these is a Lucene index, each Lucene index has some number of segments, each segment consists of a number of files. If you were to look at all these files you'd see you hit that 16384 number. Also, directories count as do open socket connections. ## Otis ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) Search Analytics - [Cloud Monitoring Tools & Services | Sematext](http://sematext.com/search-analytics/index.html) On Tuesday, April 9, 2013 8:02:19 AM UTC-4, vamsi krishna wrote: > Getting the below warning messages continuously. Not sure what should be > done. Saw some of the relevant posts asking to increase the number of file > descriptors. > > How to do the same? > > Even if I increase now, Will I encounter the same issue on addition of new > indices. > (presently working with around 400 indices, 6 shards and 1 replica). The > number of indices tend to grow more. > > [03:58:24,165][WARN][cluster.action.shard] [node1] received shard > failed for [index9][2], node[node\_hash3], [P], s[INITIALIZING], reason > [Failed to start shard, message > [IndexShardGatewayRecoveryException[[index9][2] failed recovery]; nested: > EngineCreationFailureException[[index9][2] failed to open reader on > writer]; > nested: > FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index9/2/index/segments\_1 > > (Too many open files)]; ]] > [03:58:24,166][WARN][cluster.action.shard] [node1] received shard > failed for [index15][0], node[node\_hash2], [P], s[INITIALIZING], reason > [Failed to create shard, message [IndexShardCreationException[[index15][0] > failed to create shard]; nested: IOException[directory > '/data/elasticsearch/whatever/nodes/0/indices/index15/0/index' exists and > is > a directory, but cannot be listed: list() returned null]; ]] > [03:58:24,195][WARN][cluster.action.shard] [node1] received shard > failed for [index16][3], node[node\_hash3], [P], s[INITIALIZING], reason > [Failed to start shard, message > [IndexShardGatewayRecoveryException[[index16][3] failed recovery]; nested: > EngineCreationFailureException[[index16][3] failed to open reader on > writer]; nested: > FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index16/3/index/segments\_1 > > (Too many open files)]; ]] > [03:58:24,196][WARN][cluster.action.shard] [node1] received shard > failed for [index17][0], node[node\_hash3], [P], s[INITIALIZING], reason > [Failed to start shard, message > [IndexShardGatewayRecoveryException[[index17][0] failed recovery]; nested: > EngineCreationFailureException[[index17][0] failed to open reader on > writer]; nested: > FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index17/0/index/segments\_1 > > (Too many open files)]; ]] > [03:58:24,198][WARN][cluster.action.shard] [node1] received shard > failed for [index21][4], node[node\_hash3], [P], s[INITIALIZING], reason > [Failed to start shard, message > [IndexShardGatewayRecoveryException[[index21][4] failed recovery]; nested: > EngineCreationFailureException[[index21][4] failed to create engine]; > nested: LockReleaseFailedException[Cannot forcefully unlock a NativeFSLock > which is held by another indexer component: > /data/elasticsearch/whatever/nodes/0/indices/index21/4/index/write.lock]; > ]] > > Output of nodes api curl -XGET > '[http://localhost:9200/\_nodes?os=true&process=true&pretty=true](http://localhost:9200/_nodes?os=true&process=true&pretty=true)' > > { > "ok" : true, > "cluster\_name" : "whatever", > "nodes" : { > "node\_hash1" : { > "name" : "node1", > "transport\_address" : "transportip1", > "hostname" : "myhostip1", > "version" : "0.20.4", > "http\_address" : "httpip1", > "attributes" : { > "data" : "false", > "master" : "true" > }, > "os" : { > "refresh\_interval" : 1000, > "available\_processors" : 8, > "cpu" : { > "vendor" : "Intel", > "model" : "Xeon", > "mhz" : 2133, > "total\_cores" : 8, > "total\_sockets" : 8, > "cores\_per\_socket" : 16, > "cache\_size" : "4kb", > "cache\_size\_in\_bytes" : 4096 > }, > "mem" : { > "total" : "7gb", > "total\_in\_bytes" : 7516336128 > }, > "swap" : { > "total" : "30gb", > "total\_in\_bytes" : 32218378240 > } > }, > "process" : { > "refresh\_interval" : 1000, > "id" : 26188, > "max\_file\_descriptors" : 16384 > } > }, > "node\_hash2" : { > "name" : "node2", > "transport\_address" : "transportip2", > "hostname" : "myhostip2", > "version" : "0.20.4", > "attributes" : { > "master" : "false" > }, > "os" : { > "refresh\_interval" : 1000, > "available\_processors" : 4, > "cpu" : { > "vendor" : "Intel", > "model" : "Xeon", > "mhz" : 2400, > "total\_cores" : 4, > "total\_sockets" : 4, > "cores\_per\_socket" : 32, > "cache\_size" : "20kb", > "cache\_size\_in\_bytes" : 20480 > }, > "mem" : { > "total" : "34.1gb", > "total\_in\_bytes" : 36700303360 > }, > "swap" : { > "total" : "0b", > "total\_in\_bytes" : 0 > } > }, > "process" : { > "refresh\_interval" : 1000, > "id" : 24883, > "max\_file\_descriptors" : 16384 > } > }, > "node\_hash3" : { > "name" : "node3", > "transport\_address" : "transportip3", > "hostname" : "myhostip3", > "version" : "0.20.4", > "attributes" : { > "master" : "false" > }, > "os" : { > "refresh\_interval" : 1000, > "available\_processors" : 4, > "cpu" : { > "vendor" : "Intel", > "model" : "Xeon", > "mhz" : 2666, > "total\_cores" : 4, > "total\_sockets" : 4, > "cores\_per\_socket" : 16, > "cache\_size" : "8kb", > "cache\_size\_in\_bytes" : 8192 > }, > "mem" : { > "total" : "34.1gb", > "total\_in\_bytes" : 36700303360 > }, > "swap" : { > "total" : "0b", > "total\_in\_bytes" : 0 > } > }, > "process" : { > "refresh\_interval" : 1000, > "id" : 25328, > "max\_file\_descriptors" : 16384 > } > } > } > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Too-many-open-files-warning-tp4033111.html](http://elasticsearch-users.115913.n3.nabble.com/Too-many-open-files-warning-tp4033111.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [April 11, 2013, 12:37pm UTC](https://discuss.elastic.co/t/too-many-open-files-warning/11503/3 "2013-04-11T12:37:24Z")
my advice here is: set the file handle limit to "unlimited" and you are good here simon On Wednesday, April 10, 2013 7:19:41 PM UTC+2, Otis Gospodnetic wrote: > Hello, > > Note this part: > > ``` > "process" : { > "refresh_interval" : 1000, > "id" : 25328, > "max_file_descriptors" : *16384* > } > > ``` > > Don't be afraid to increase quadruple this. Your 600 indices each have > some number of shards, each shard some number of replicas, each of these is > a Lucene index, each Lucene index has some number of segments, each segment > consists of a number of files. If you were to look at all these files > you'd see you hit that 16384 number. Also, directories count as do open > socket connections. > > ## Otis > > ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) > Search Analytics - [Cloud Monitoring Tools & Services | Sematext](http://sematext.com/search-analytics/index.html) > > On Tuesday, April 9, 2013 8:02:19 AM UTC-4, vamsi krishna wrote: > > > Getting the below warning messages continuously. Not sure what should be > > done. Saw some of the relevant posts asking to increase the number of > > file > > descriptors. > > > > How to do the same? > > > > Even if I increase now, Will I encounter the same issue on addition of > > new > > indices. > > (presently working with around 400 indices, 6 shards and 1 replica). The > > number of indices tend to grow more. > > > > [03:58:24,165][WARN][cluster.action.shard] [node1] received shard > > failed for [index9][2], node[node\_hash3], [P], s[INITIALIZING], reason > > [Failed to start shard, message > > [IndexShardGatewayRecoveryException[[index9][2] failed recovery]; nested: > > EngineCreationFailureException[[index9][2] failed to open reader on > > writer]; > > nested: > > FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index9/2/index/segments\_1 > > > > (Too many open files)]; ]] > > [03:58:24,166][WARN][cluster.action.shard] [node1] received shard > > failed for [index15][0], node[node\_hash2], [P], s[INITIALIZING], reason > > [Failed to create shard, message > > [IndexShardCreationException[[index15][0] > > failed to create shard]; nested: IOException[directory > > '/data/elasticsearch/whatever/nodes/0/indices/index15/0/index' exists and > > is > > a directory, but cannot be listed: list() returned null]; ]] > > [03:58:24,195][WARN][cluster.action.shard] [node1] received shard > > failed for [index16][3], node[node\_hash3], [P], s[INITIALIZING], reason > > [Failed to start shard, message > > [IndexShardGatewayRecoveryException[[index16][3] failed recovery]; > > nested: > > EngineCreationFailureException[[index16][3] failed to open reader on > > writer]; nested: > > FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index16/3/index/segments\_1 > > > > (Too many open files)]; ]] > > [03:58:24,196][WARN][cluster.action.shard] [node1] received shard > > failed for [index17][0], node[node\_hash3], [P], s[INITIALIZING], reason > > [Failed to start shard, message > > [IndexShardGatewayRecoveryException[[index17][0] failed recovery]; > > nested: > > EngineCreationFailureException[[index17][0] failed to open reader on > > writer]; nested: > > FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index17/0/index/segments\_1 > > > > (Too many open files)]; ]] > > [03:58:24,198][WARN][cluster.action.shard] [node1] received shard > > failed for [index21][4], node[node\_hash3], [P], s[INITIALIZING], reason > > [Failed to start shard, message > > [IndexShardGatewayRecoveryException[[index21][4] failed recovery]; > > nested: > > EngineCreationFailureException[[index21][4] failed to create engine]; > > nested: LockReleaseFailedException[Cannot forcefully unlock a > > NativeFSLock > > which is held by another indexer component: > > /data/elasticsearch/whatever/nodes/0/indices/index21/4/index/write.lock]; > > ]] > > > > Output of nodes api curl -XGET > > '[http://localhost:9200/\_nodes?os=true&process=true&pretty=true](http://localhost:9200/_nodes?os=true&process=true&pretty=true)' > > > > { > > "ok" : true, > > "cluster\_name" : "whatever", > > "nodes" : { > > "node\_hash1" : { > > "name" : "node1", > > "transport\_address" : "transportip1", > > "hostname" : "myhostip1", > > "version" : "0.20.4", > > "http\_address" : "httpip1", > > "attributes" : { > > "data" : "false", > > "master" : "true" > > }, > > "os" : { > > "refresh\_interval" : 1000, > > "available\_processors" : 8, > > "cpu" : { > > "vendor" : "Intel", > > "model" : "Xeon", > > "mhz" : 2133, > > "total\_cores" : 8, > > "total\_sockets" : 8, > > "cores\_per\_socket" : 16, > > "cache\_size" : "4kb", > > "cache\_size\_in\_bytes" : 4096 > > }, > > "mem" : { > > "total" : "7gb", > > "total\_in\_bytes" : 7516336128 > > }, > > "swap" : { > > "total" : "30gb", > > "total\_in\_bytes" : 32218378240 > > } > > }, > > "process" : { > > "refresh\_interval" : 1000, > > "id" : 26188, > > "max\_file\_descriptors" : 16384 > > } > > }, > > "node\_hash2" : { > > "name" : "node2", > > "transport\_address" : "transportip2", > > "hostname" : "myhostip2", > > "version" : "0.20.4", > > "attributes" : { > > "master" : "false" > > }, > > "os" : { > > "refresh\_interval" : 1000, > > "available\_processors" : 4, > > "cpu" : { > > "vendor" : "Intel", > > "model" : "Xeon", > > "mhz" : 2400, > > "total\_cores" : 4, > > "total\_sockets" : 4, > > "cores\_per\_socket" : 32, > > "cache\_size" : "20kb", > > "cache\_size\_in\_bytes" : 20480 > > }, > > "mem" : { > > "total" : "34.1gb", > > "total\_in\_bytes" : 36700303360 > > }, > > "swap" : { > > "total" : "0b", > > "total\_in\_bytes" : 0 > > } > > }, > > "process" : { > > "refresh\_interval" : 1000, > > "id" : 24883, > > "max\_file\_descriptors" : 16384 > > } > > }, > > "node\_hash3" : { > > "name" : "node3", > > "transport\_address" : "transportip3", > > "hostname" : "myhostip3", > > "version" : "0.20.4", > > "attributes" : { > > "master" : "false" > > }, > > "os" : { > > "refresh\_interval" : 1000, > > "available\_processors" : 4, > > "cpu" : { > > "vendor" : "Intel", > > "model" : "Xeon", > > "mhz" : 2666, > > "total\_cores" : 4, > > "total\_sockets" : 4, > > "cores\_per\_socket" : 16, > > "cache\_size" : "8kb", > > "cache\_size\_in\_bytes" : 8192 > > }, > > "mem" : { > > "total" : "34.1gb", > > "total\_in\_bytes" : 36700303360 > > }, > > "swap" : { > > "total" : "0b", > > "total\_in\_bytes" : 0 > > } > > }, > > "process" : { > > "refresh\_interval" : 1000, > > "id" : 25328, > > "max\_file\_descriptors" : 16384 > > } > > } > > } > > > > -- > > View this message in context: > > [http://elasticsearch-users.115913.n3.nabble.com/Too-many-open-files-warning-tp4033111.html](http://elasticsearch-users.115913.n3.nabble.com/Too-many-open-files-warning-tp4033111.html) > > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [April 12, 2013, 10:08am UTC](https://discuss.elastic.co/t/too-many-open-files-warning/11503/4 "2013-04-12T10:08:24Z")
On Thu, 2013-04-11 at 05:37 -0700, simonw wrote: > my advice here is: set the file handle limit to "unlimited" and you > are good here On some (all? many?) versions of linux you can't set it to unlimited, in which case set it to a high number, eg 128k clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![ovamsikrishna](https://avatars.discourse-cdn.com/v4/letter/o/e8c25b/32.png) [@ovamsikrishna](https://discuss.elastic.co/u/ovamsikrishna)\ **Post date:** [April 18, 2013, 8:35am UTC](https://discuss.elastic.co/t/too-many-open-files-warning/11503/5 "2013-04-18T08:35:49Z")
Thanks everyone for the replies. @simon and @Clinton - I have thought of that before. But, would that have any performance issue? (setting number of open file descriptors to unlimited at OS level) ---
**Author:** ![Anurag](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anurag/32/2286_2.png) [@Anurag](https://discuss.elastic.co/u/Anurag)\ **Post date:** [April 21, 2013, 10:40am UTC](https://discuss.elastic.co/t/too-many-open-files-warning/11503/6 "2013-04-21T10:40:03Z")
it can be set on a per-user or per-group basis too. On Thu, Apr 18, 2013 at 2:05 PM, ovamsikrishna [ovamsikrishna@gmail.com](mailto:ovamsikrishna@gmail.com)wrote: > Thanks everyone for the replies. > > @simon and @Clinton - I have thought of that before. But, would that have > any performance issue? (setting number of open file descriptors to > unlimited > at OS level) > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Too-many-open-files-warning-tp4033111p4033572.html](http://elasticsearch-users.115913.n3.nabble.com/Too-many-open-files-warning-tp4033111p4033572.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Anurag \<0xB20A82C1\> [http://web.gnuer.org/blog/](http://web.gnuer.org/blog/) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [April 21, 2013, 12:50pm UTC](https://discuss.elastic.co/t/too-many-open-files-warning/11503/7 "2013-04-21T12:50:10Z")
I always do this... (unlimited or very high) The problem is that most of the time you run into the high num of open files problem it only lasts for a second or so and then the number goes down quickly. Don't worry what you don't want is running our of file handles! I think its safe to set unlimited. simon On Thursday, April 18, 2013 10:35:49 AM UTC+2, vamsi krishna wrote: > Thanks everyone for the replies. > > @simon and @Clinton - I have thought of that before. But, would that have > any performance issue? (setting number of open file descriptors to > unlimited > at OS level) > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Too-many-open-files-warning-tp4033111p4033572.html](http://elasticsearch-users.115913.n3.nabble.com/Too-many-open-files-warning-tp4033111p4033572.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [April 21, 2013, 4:58pm UTC](https://discuss.elastic.co/t/too-many-open-files-warning/11503/8 "2013-04-21T16:58:49Z")
The Linux resource limit for ulimit can not exceed the hard limit given by the file descriptor table of the kernel. Under Linux 2.0, back in 1998, you had 256 file descriptors in a process, and a maximum of 1024 for the system, and if you wanted more, you had to hack the kernel header files und recompile 🙂 [https://www.redhat.com/archives/redhat-list/1998-November/msg03103.html](https://www.redhat.com/archives/redhat-list/1998-November/msg03103.html) Under Linux 2.2 the resource limit was raised to 1024, and for 3.0 it was raised to 4096. [http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=0ac1ee0bfec2a4ad118f907ce586d0dfd8db7641](http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=0ac1ee0bfec2a4ad118f907ce586d0dfd8db7641) Today, Linux has a hardcoded limit of 1024\*1024 file descriptors, which is later raised to "unlimited" (MAXINT) see [http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/fs/file.c](http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/fs/file.c) In earlier days, the Linux kernel reserved the space for the file descriptors in a static array. So it was a ridiculous waste of memory if you had set this limit too high. You can check your kernel file limit with cat /proc/sys/fs/file-max and the current situation of the file descriptor allocation with cat /proc/sys/fs/file-nr (the numbers are: currently active, currently inactive, maximum) Just to note, there is a difference between soft/hard resource limit, see > **[5.10. Kernel Red Hat Enterprise Linux 6 | Red Hat Customer Portal](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/6.4_technical_notes/kernel_issues)** > > Access Red Hat’s knowledge, guidance, and support through your subscription. " Applications are increasingly using more than 1024 file descriptors. It is not recommended to increase the default soft limit of file descriptors because it may break applications that use the select() call. However, it is safe to increase the default hard limit; that way, applications requiring a large amount of file descriptors can increase their soft limit without needing root privileges and without any user intervention." Dan Kegel documented the situation of file handles in his C10k paper some time ago [The C10K problem](http://www.kegel.com/c10k.html#limits.filehandles) To make things more unpredictable, the built-in Linux kernel limit and the default resource limit value for ulimit depend on the Linux distributor. For example, Redhat Enterprise Linux still keeps the ulimit setting of 1024. For Elasticsearch, I have a rule of thumb: each shard (each Lucene index) takes ~150 file descriptors (files, pipes, epolls, sockets, etc. maybe more, up to 400 when segment merging gives peaks of file descriptor consumption, which can be decreased by switching to the compound index format). That's the reason why ES works with unmodified 1024 settings on a single machine with the 5 shard default, but as soon as more nodes and replica shards and more segment merging is going on, the 1024 limit will be exceeded. Jörg Am 12.04.13 12:08, schrieb Clinton Gormley: > On Thu, 2013-04-11 at 05:37 -0700, simonw wrote: > > > my advice here is: set the file handle limit to "unlimited" and you > > are good here > > On some (all? many?) versions of linux you can't set it to unlimited, in > > which case set it to a high number, eg 128k > > clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![ovamsikrishna](https://avatars.discourse-cdn.com/v4/letter/o/e8c25b/32.png) [@ovamsikrishna](https://discuss.elastic.co/u/ovamsikrishna)\ **Post date:** [April 22, 2013, 5:33am UTC](https://discuss.elastic.co/t/too-many-open-files-warning/11503/9 "2013-04-22T05:33:46Z")
Thanks @simonw-2... In my case, I never saw it going down. It keeps on increasing. (May be because the number of indices keeps on increasing in my case) I'm already using the compound\_format setting. That is the reason, I'm bit worried about the performance. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:40am UTC](https://discuss.elastic.co/t/too-many-open-files-warning/11503/10 "2017-07-06T02:40:15Z")