# Indices.memory.index\_buffer\_size **URL:** \ **Category:** Elasticsearch\ **Created:** [August 24, 2014, 5:12pm UTC](https://discuss.elastic.co/t/indices-memory-index-buffer-size/19426 "2014-08-24T17:12:55Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Yongtao\_You](https://avatars.discourse-cdn.com/v4/letter/y/dbc845/32.png) [@Yongtao\_You](https://discuss.elastic.co/u/Yongtao_You)\ **Post date:** [August 24, 2014, 5:12pm UTC](https://discuss.elastic.co/t/indices-memory-index-buffer-size/19426/1 "2014-08-24T17:12:55Z")
Hi, Is the "indices.memory.index\_buffer\_size" configuration a cluster wide configuration or per node configuration? Do I need to set it on every node? Or just the master (eligible) node? Thanks. Yongtao -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 24, 2014, 10:12pm UTC](https://discuss.elastic.co/t/indices-memory-index-buffer-size/19426/2 "2014-08-24T22:12:40Z")
> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. states "It is a global setting that bubbles down to all the different shards allocated on a specific node." Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 25 August 2014 03:12, Yongtao You [yongtao.you@gmail.com](mailto:yongtao.you@gmail.com) wrote: > Hi, > > Is the "indices.memory.index\_buffer\_size" configuration a cluster wide > configuration or per node configuration? Do I need to set it on every node? > Or just the master (eligible) node? > > Thanks. > Yongtao > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624ZrhCzaXc4qAHeiOH8RKr6bKB-oRGH8YDFpz%3D34m29Y3Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624ZrhCzaXc4qAHeiOH8RKr6bKB-oRGH8YDFpz%3D34m29Y3Q%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Yongtao\_You](https://avatars.discourse-cdn.com/v4/letter/y/dbc845/32.png) [@Yongtao\_You](https://discuss.elastic.co/u/Yongtao_You)\ **Post date:** [August 26, 2014, 6:07pm UTC](https://discuss.elastic.co/t/indices-memory-index-buffer-size/19426/3 "2014-08-26T18:07:28Z")
Thanks Mark. What confuses me are "global setting" (which suggests cluster-wide setting) and "on a specific node" (which suggests node level setting). I could just try it out, but it's hard to tell if the setting worked or not. ☹ On Sunday, August 24, 2014 3:13:17 PM UTC-7, Mark Walkom wrote: > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-indices.html) > states "It is a global setting that bubbles down to all the different > shards allocated on a specific node." > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) \ > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > On 25 August 2014 03:12, Yongtao You \<[yongt...@gmail.com](mailto:yongt...@gmail.com) \\> > wrote: > > > Hi, > > > > Is the "indices.memory.index\_buffer\_size" configuration a cluster wide > > configuration or per node configuration? Do I need to set it on every node? > > Or just the master (eligible) node? > > > > Thanks. > > Yongtao > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d76f4c67-9250-4ab9-b02d-0f0c78b33be6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d76f4c67-9250-4ab9-b02d-0f0c78b33be6%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [August 26, 2014, 6:15pm UTC](https://discuss.elastic.co/t/indices-memory-index-buffer-size/19426/4 "2014-08-26T18:15:22Z")
I just looked at this code! Its a setting that you set globally at the cluster level. It takes effect per node. What that means is that for every "active" shard on each the node gets an equal share of that much space. "Active" means has been written to in the past six minutes or so. When a node first starts all shards are active assumed active and those that are not updated at all lose active status after the timeout. You can watch the little dance it does by setting index.engine.internal: DEBUG in logging.yml. Now - I'm not actually sure how important a setting it it is. I opened [indices.memory.index\_buffer\_size is spread evenly across active shards · Issue #7441 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/7441) to get suggest allowing better spreading it around. Mike'll probably close it if spreading it around wouldn't really help things much. Nik On Tue, Aug 26, 2014 at 2:07 PM, Yongtao You [yongtao.you@gmail.com](mailto:yongtao.you@gmail.com) wrote: > Thanks Mark. > > What confuses me are "global setting" (which suggests cluster-wide > setting) and "on a specific node" (which suggests node level setting). I > could just try it out, but it's hard to tell if the setting worked or not. > ☹ > > On Sunday, August 24, 2014 3:13:17 PM UTC-7, Mark Walkom wrote: > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/) > > reference/current/modules-indices.html states "It is a global setting > > that bubbles down to all the different shards allocated on a specific node." > > > > Regards, > > Mark Walkom > > > > Infrastructure Engineer > > Campaign Monitor > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > On 25 August 2014 03:12, Yongtao You [yongt...@gmail.com](mailto:yongt...@gmail.com) wrote: > > > > > Hi, > > > > > > Is the "indices.memory.index\_buffer\_size" configuration a cluster wide > > > configuration or per node configuration? Do I need to set it on every node? > > > Or just the master (eligible) node? > > > > > > Thanks. > > > Yongtao > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c% > > > [40googlegroups.com](http://40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/d76f4c67-9250-4ab9-b02d-0f0c78b33be6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d76f4c67-9250-4ab9-b02d-0f0c78b33be6%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/d76f4c67-9250-4ab9-b02d-0f0c78b33be6%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d76f4c67-9250-4ab9-b02d-0f0c78b33be6%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1CmkjPAPJns3PjCmsFicu8KYV0DRjv9T2qacx636sy7g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1CmkjPAPJns3PjCmsFicu8KYV0DRjv9T2qacx636sy7g%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![mikemccand](https://avatars.discourse-cdn.com/v4/letter/m/f04885/32.png) [@mikemccand](https://discuss.elastic.co/u/mikemccand)\ **Post date:** [August 26, 2014, 8:31pm UTC](https://discuss.elastic.co/t/indices-memory-index-buffer-size/19426/5 "2014-08-26T20:31:44Z")
See also [Add `segments.index_writer_max_memory` to stats by mikemccand · Pull Request #7440 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/pull/7440) (will be in 1.4.0) which returns the actual RAM buffer size assigned to that shard by the "little dance". Mike McCandless [http://blog.mikemccandless.com](http://blog.mikemccandless.com) On Tue, Aug 26, 2014 at 2:15 PM, Nikolas Everett [nik9000@gmail.com](mailto:nik9000@gmail.com) wrote: > I just looked at this code! > > Its a setting that you set globally at the cluster level. It takes effect > per node. What that means is that for every "active" shard on each the > node gets an equal share of that much space. "Active" means has been > written to in the past six minutes or so. When a node first starts all > shards are active assumed active and those that are not updated at all lose > active status after the timeout. You can watch the little dance it does by > setting > index.engine.internal: DEBUG > in logging.yml. > > Now - I'm not actually sure how important a setting it it is. I opened > [indices.memory.index\_buffer\_size is spread evenly across active shards · Issue #7441 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/7441) to get suggest > allowing better spreading it around. Mike'll probably close it if > spreading it around wouldn't really help things much. > > Nik > > On Tue, Aug 26, 2014 at 2:07 PM, Yongtao You [yongtao.you@gmail.com](mailto:yongtao.you@gmail.com) > wrote: > > > Thanks Mark. > > > > What confuses me are "global setting" (which suggests cluster-wide > > setting) and "on a specific node" (which suggests node level setting). I > > could just try it out, but it's hard to tell if the setting worked or not. > > ☹ > > > > On Sunday, August 24, 2014 3:13:17 PM UTC-7, Mark Walkom wrote: > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/) > > > reference/current/modules-indices.html states "It is a global setting > > > that bubbles down to all the different shards allocated on a specific node." > > > > > > Regards, > > > Mark Walkom > > > > > > Infrastructure Engineer > > > Campaign Monitor > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > > > On 25 August 2014 03:12, Yongtao You [yongt...@gmail.com](mailto:yongt...@gmail.com) wrote: > > > > > > > Hi, > > > > > > > > Is the "indices.memory.index\_buffer\_size" configuration a cluster wide > > > > configuration or per node configuration? Do I need to set it on every node? > > > > Or just the master (eligible) node? > > > > > > > > Thanks. > > > > Yongtao > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c% > > > > [40googlegroups.com](http://40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f67e3a30-521c-4c13-8620-c79133cea01c%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/d76f4c67-9250-4ab9-b02d-0f0c78b33be6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d76f4c67-9250-4ab9-b02d-0f0c78b33be6%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/d76f4c67-9250-4ab9-b02d-0f0c78b33be6%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d76f4c67-9250-4ab9-b02d-0f0c78b33be6%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1CmkjPAPJns3PjCmsFicu8KYV0DRjv9T2qacx636sy7g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1CmkjPAPJns3PjCmsFicu8KYV0DRjv9T2qacx636sy7g%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1CmkjPAPJns3PjCmsFicu8KYV0DRjv9T2qacx636sy7g%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1CmkjPAPJns3PjCmsFicu8KYV0DRjv9T2qacx636sy7g%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAD7smRcwFh-qCtu6B15Ni9KjzCYVojXtc4KTzMc%2Be1BVHZ%3D-Bw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAD7smRcwFh-qCtu6B15Ni9KjzCYVojXtc4KTzMc%2Be1BVHZ%3D-Bw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:06am UTC](https://discuss.elastic.co/t/indices-memory-index-buffer-size/19426/6 "2017-07-06T01:06:15Z")