# Many number of open files **URL:** \ **Category:** Elasticsearch\ **Created:** [June 13, 2012, 6:47pm UTC](https://discuss.elastic.co/t/many-number-of-open-files/8086 "2012-06-13T18:47:18Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![arta](https://avatars.discourse-cdn.com/v4/letter/a/aca169/32.png) [@arta](https://discuss.elastic.co/u/arta)\ **Post date:** [June 13, 2012, 6:47pm UTC](https://discuss.elastic.co/t/many-number-of-open-files/8086/1 "2012-06-13T18:47:18Z")
Hi, In my experiment, I indexed about 80 million documents and I observed the number of open files being high. My setup is as follows: - number of nodes in the cluster: 2 - number of indices: 32 - number of shards (for each index): 32 - number of replicas: 1 The number of open files observed were: node1: 47508 node2: 47412 The number of open files was increasing as time went on. (I eventually had out of memory exception, so I had to stop my experiment there.) With my setup, it creates 32 indices x 32 shards x 2 replicas / 2 nodes = 1024 Lucene indices per node. I know the number of shards is too big for this environment, but this was an experiment. Is about 50,000 open files for about 1000 Lucene indices as expected? Is the number somewhat proportional to number of Lucene indices? My observation is that the number of files in each Lucene index directory varies (100 to 200). I think, correct me if I'm wrong, the number increase as more documents are indexed and decreases when optimization runs. In my current system setup, I set the max limit of number of open files per process to 100,000. Can I say this is ok setup for about 1000 Lucene indices? Thanks for your help. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [June 13, 2012, 6:51pm UTC](https://discuss.elastic.co/t/many-number-of-open-files/8086/2 "2012-06-13T18:51:51Z")
The more shards (Luceen index) you have will increase the number of open files required. The number of open files each Lucene index needs depends internally on the number of segments it has, which depend in turn on the merge policy settings. On Wed, Jun 13, 2012 at 8:47 PM, arta [artasano@sbcglobal.net](mailto:artasano@sbcglobal.net) wrote: > Hi, > In my experiment, I indexed about 80 million documents and I observed the > number of open files being high. > My setup is as follows: > > - number of nodes in the cluster: 2 > - number of indices: 32 > - number of shards (for each index): 32 > - number of replicas: 1 > The number of open files observed were: > node1: 47508 > node2: 47412 > > The number of open files was increasing as time went on. > (I eventually had out of memory exception, so I had to stop my experiment > there.) > > With my setup, it creates 32 indices x 32 shards x 2 replicas / 2 nodes = > 1024 Lucene indices per node. > I know the number of shards is too big for this environment, but this was > an > experiment. > > Is about 50,000 open files for about 1000 Lucene indices as expected? > > Is the number somewhat proportional to number of Lucene indices? > My observation is that the number of files in each Lucene index directory > varies (100 to 200). > I think, correct me if I'm wrong, the number increase as more documents are > indexed and decreases when optimization runs. > In my current system setup, I set the max limit of number of open files per > process to 100,000. > Can I say this is ok setup for about 1000 Lucene indices? > > Thanks for your help. > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Many-number-of-open-files-tp4019243.html](http://elasticsearch-users.115913.n3.nabble.com/Many-number-of-open-files-tp4019243.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:24am UTC](https://discuss.elastic.co/t/many-number-of-open-files/8086/3 "2017-07-06T03:24:24Z")