# What configuration is available to control MemoryMapDirectory **URL:** \ **Category:** Elasticsearch\ **Created:** [March 14, 2015, 12:03am UTC](https://discuss.elastic.co/t/what-configuration-is-available-to-control-memorymapdirectory/22665 "2015-03-14T00:03:34Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![lpoole](https://avatars.discourse-cdn.com/v4/letter/l/d26b3c/32.png) [@lpoole](https://discuss.elastic.co/u/lpoole)\ **Post date:** [March 14, 2015, 12:03am UTC](https://discuss.elastic.co/t/what-configuration-is-available-to-control-memorymapdirectory/22665/1 "2015-03-14T00:03:34Z")
Hey guys, We're running into some problems under heavy write, nominal read volume when the Lucene memory mapped files have exhausted available physical memory, and segments from disk must be paged into memory. Are there any configs available to control how much physical memory is available to MemoryMapDirectory? Thanks! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 14, 2015, 6:27am UTC](https://discuss.elastic.co/t/what-configuration-is-available-to-control-memorymapdirectory/22665/2 "2015-03-14T06:27:27Z")
There may not be much you can do here beyond giving the system more RAM and moving to SSD. How bad is the problem? On 13 March 2015 at 17:03, Lindsey Poole [lpoole@gmail.com](mailto:lpoole@gmail.com) wrote: > Hey guys, > > We're running into some problems under heavy write, nominal read volume > when the Lucene memory mapped files have exhausted available physical > memory, and segments from disk must be paged into memory. > > Are there any configs available to control how much physical memory is > available to MemoryMapDirectory? > > Thanks! > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8zSG55xY6SUF%2B9iB%3DTo4m%2Bv8VxgX2iUe8%3DaoqSPmBxGw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8zSG55xY6SUF%2B9iB%3DTo4m%2Bv8VxgX2iUe8%3DaoqSPmBxGw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 14, 2015, 11:43am UTC](https://discuss.elastic.co/t/what-configuration-is-available-to-control-memorymapdirectory/22665/3 "2015-03-14T11:43:03Z")
You may try limit direct memory on JVM level by using -XX:MaxDirectMemorySize (default is unlimited). See also ES\_DIRECT\_SIZE in > **[Running as a service on Linux | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-service.html#_linux)** I recommend at least 2GB Jörg On Sat, Mar 14, 2015 at 1:03 AM, Lindsey Poole [lpoole@gmail.com](mailto:lpoole@gmail.com) wrote: > Hey guys, > > We're running into some problems under heavy write, nominal read volume > when the Lucene memory mapped files have exhausted available physical > memory, and segments from disk must be paged into memory. > > Are there any configs available to control how much physical memory is > available to MemoryMapDirectory? > > Thanks! > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH1q7WdOhPWo9D8vdCDVU97epZhATOU6qZyt\_sVS\_ySeg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH1q7WdOhPWo9D8vdCDVU97epZhATOU6qZyt_sVS_ySeg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![lpoole](https://avatars.discourse-cdn.com/v4/letter/l/d26b3c/32.png) [@lpoole](https://discuss.elastic.co/u/lpoole)\ **Post date:** [March 14, 2015, 4:04pm UTC](https://discuss.elastic.co/t/what-configuration-is-available-to-control-memorymapdirectory/22665/4 "2015-03-14T16:04:34Z")
I did see the ES\_DIRECT\_SIZE, but it seems to be ineffective. I will try setting -XX:MaxDirectMemorySize directly. On Saturday, March 14, 2015 at 4:43:22 AM UTC-7, Jörg Prante wrote: > You may try limit direct memory on JVM level by > using -XX:MaxDirectMemorySize (default is unlimited). See also > ES\_DIRECT\_SIZE in > [Running as a service on Linux | Elasticsearch Guide [8.11] | Elastic](http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-service.html#_linux) > > I recommend at least 2GB > > Jörg > > On Sat, Mar 14, 2015 at 1:03 AM, Lindsey Poole \<[lpo...@gmail.com](mailto:lpo...@gmail.com) > \\> wrote: > > > Hey guys, > > > > We're running into some problems under heavy write, nominal read volume > > when the Lucene memory mapped files have exhausted available physical > > memory, and segments from disk must be paged into memory. > > > > Are there any configs available to control how much physical memory is > > available to MemoryMapDirectory? > > > > Thanks! > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0f540631-52f0-48d3-b33a-855c54d8ce94%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0f540631-52f0-48d3-b33a-855c54d8ce94%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![lpoole](https://avatars.discourse-cdn.com/v4/letter/l/d26b3c/32.png) [@lpoole](https://discuss.elastic.co/u/lpoole)\ **Post date:** [March 14, 2015, 4:06pm UTC](https://discuss.elastic.co/t/what-configuration-is-available-to-control-memorymapdirectory/22665/5 "2015-03-14T16:06:02Z")
btw - we're on EC2 I2-4xl hosts, so we have ~120g ram and SSDs. On Saturday, March 14, 2015 at 9:04:34 AM UTC-7, Lindsey Poole wrote: > I did see the ES\_DIRECT\_SIZE, but it seems to be ineffective. > > I will try setting -XX:MaxDirectMemorySize directly. > > On Saturday, March 14, 2015 at 4:43:22 AM UTC-7, Jörg Prante wrote: > > > You may try limit direct memory on JVM level by > > using -XX:MaxDirectMemorySize (default is unlimited). See also > > ES\_DIRECT\_SIZE in > > [Running as a service on Linux | Elasticsearch Guide [8.11] | Elastic](http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-service.html#_linux) > > > > I recommend at least 2GB > > > > Jörg > > > > On Sat, Mar 14, 2015 at 1:03 AM, Lindsey Poole [lpo...@gmail.com](mailto:lpo...@gmail.com) wrote: > > > > > Hey guys, > > > > > > We're running into some problems under heavy write, nominal read volume > > > when the Lucene memory mapped files have exhausted available physical > > > memory, and segments from disk must be paged into memory. > > > > > > Are there any configs available to control how much physical memory is > > > available to MemoryMapDirectory? > > > > > > Thanks! > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 14, 2015, 5:12pm UTC](https://discuss.elastic.co/t/what-configuration-is-available-to-control-memorymapdirectory/22665/6 "2015-03-14T17:12:11Z")
I'm out - no experience with EC2. I avoid foreign servers at all cost. Maybe 120G RAM is affected by swap/memory overcommit. Do not forget to check memlock and memory ballooning. The chances are few you can control host settings as a guest in a virtual server environment. Jörg On Sat, Mar 14, 2015 at 5:06 PM, Lindsey Poole [lpoole@gmail.com](mailto:lpoole@gmail.com) wrote: > btw - we're on EC2 I2-4xl hosts, so we have ~120g ram and SSDs. > > On Saturday, March 14, 2015 at 9:04:34 AM UTC-7, Lindsey Poole wrote: > > > I did see the ES\_DIRECT\_SIZE, but it seems to be ineffective. > > > > I will try setting -XX:MaxDirectMemorySize directly. > > > > On Saturday, March 14, 2015 at 4:43:22 AM UTC-7, Jörg Prante wrote: > > > > > You may try limit direct memory on JVM level by > > > using -XX:MaxDirectMemorySize (default is unlimited). See also > > > ES\_DIRECT\_SIZE in [http://www.elastic.co/guide/en/elasticsearch/](http://www.elastic.co/guide/en/elasticsearch/) > > > reference/current/setup-service.html#\_linux > > > > > > I recommend at least 2GB > > > > > > Jörg > > > > > > On Sat, Mar 14, 2015 at 1:03 AM, Lindsey Poole [lpo...@gmail.com](mailto:lpo...@gmail.com) wrote: > > > > > > > Hey guys, > > > > > > > > We're running into some problems under heavy write, nominal read volume > > > > when the Lucene memory mapped files have exhausted available physical > > > > memory, and segments from disk must be paged into memory. > > > > > > > > Are there any configs available to control how much physical memory is > > > > available to MemoryMapDirectory? > > > > > > > > Thanks! > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5% > > > > [40googlegroups.com](http://40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHY\_M2BAywrG%2BaNcg59xA4\_ocph1oqE0bzba4HTqrdLqQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHY_M2BAywrG%2BaNcg59xA4_ocph1oqE0bzba4HTqrdLqQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 14, 2015, 8:05pm UTC](https://discuss.elastic.co/t/what-configuration-is-available-to-control-memorymapdirectory/22665/7 "2015-03-14T20:05:15Z")
Can you provide more info on what the error/problem is, logs might help. On 14 March 2015 at 10:12, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > I'm out - no experience with EC2. I avoid foreign servers at all cost. > Maybe 120G RAM is affected by swap/memory overcommit. Do not forget to > check memlock and memory ballooning. The chances are few you can control > host settings as a guest in a virtual server environment. > > Jörg > > On Sat, Mar 14, 2015 at 5:06 PM, Lindsey Poole [lpoole@gmail.com](mailto:lpoole@gmail.com) wrote: > > > btw - we're on EC2 I2-4xl hosts, so we have ~120g ram and SSDs. > > > > On Saturday, March 14, 2015 at 9:04:34 AM UTC-7, Lindsey Poole wrote: > > > > > I did see the ES\_DIRECT\_SIZE, but it seems to be ineffective. > > > > > > I will try setting -XX:MaxDirectMemorySize directly. > > > > > > On Saturday, March 14, 2015 at 4:43:22 AM UTC-7, Jörg Prante wrote: > > > > > > > You may try limit direct memory on JVM level by > > > > using -XX:MaxDirectMemorySize (default is unlimited). See also > > > > ES\_DIRECT\_SIZE in [http://www.elastic.co/guide/en/elasticsearch/](http://www.elastic.co/guide/en/elasticsearch/) > > > > reference/current/setup-service.html#\_linux > > > > > > > > I recommend at least 2GB > > > > > > > > Jörg > > > > > > > > On Sat, Mar 14, 2015 at 1:03 AM, Lindsey Poole [lpo...@gmail.com](mailto:lpo...@gmail.com) > > > > wrote: > > > > > > > > > Hey guys, > > > > > > > > > > We're running into some problems under heavy write, nominal read > > > > > volume when the Lucene memory mapped files have exhausted available > > > > > physical memory, and segments from disk must be paged into memory. > > > > > > > > > > Are there any configs available to control how much physical memory is > > > > > available to MemoryMapDirectory? > > > > > > > > > > Thanks! > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5% > > > > > [40googlegroups.com](http://40googlegroups.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google Groups > > > > "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send an > > > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHY\_M2BAywrG%2BaNcg59xA4\_ocph1oqE0bzba4HTqrdLqQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHY_M2BAywrG%2BaNcg59xA4_ocph1oqE0bzba4HTqrdLqQ%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHY\_M2BAywrG%2BaNcg59xA4\_ocph1oqE0bzba4HTqrdLqQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHY_M2BAywrG%2BaNcg59xA4_ocph1oqE0bzba4HTqrdLqQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-%2BAWW-v1XfUZaKvTDDgFxLv3X5gegShk8Uzi1egPa3Aw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-%2BAWW-v1XfUZaKvTDDgFxLv3X5gegShk8Uzi1egPa3Aw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![lpoole](https://avatars.discourse-cdn.com/v4/letter/l/d26b3c/32.png) [@lpoole](https://discuss.elastic.co/u/lpoole)\ **Post date:** [March 15, 2015, 9:48pm UTC](https://discuss.elastic.co/t/what-configuration-is-available-to-control-memorymapdirectory/22665/8 "2015-03-15T21:48:29Z")
Just to close this out - we disabled EC2's health checks and spent some time tuning the batch thread-pool size to prevent overrunning the cluster once the memory map cache size exceeds available physical memory. This was successful (we're restricted to a surprisingly small threadpool size of 3). Thanks. On Saturday, March 14, 2015 at 1:05:49 PM UTC-7, Mark Walkom wrote: > Can you provide more info on what the error/problem is, logs might help. > > On 14 March 2015 at 10:12, [joerg...@gmail.com](mailto:joerg...@gmail.com) \ \< > [joerg...@gmail.com](mailto:joerg...@gmail.com) \\> wrote: > > > I'm out - no experience with EC2. I avoid foreign servers at all cost. > > Maybe 120G RAM is affected by swap/memory overcommit. Do not forget to > > check memlock and memory ballooning. The chances are few you can control > > host settings as a guest in a virtual server environment. > > > > Jörg > > > > On Sat, Mar 14, 2015 at 5:06 PM, Lindsey Poole \<[lpo...@gmail.com](mailto:lpo...@gmail.com) > > \\> wrote: > > > > > btw - we're on EC2 I2-4xl hosts, so we have ~120g ram and SSDs. > > > > > > On Saturday, March 14, 2015 at 9:04:34 AM UTC-7, Lindsey Poole wrote: > > > > > > > I did see the ES\_DIRECT\_SIZE, but it seems to be ineffective. > > > > > > > > I will try setting -XX:MaxDirectMemorySize directly. > > > > > > > > On Saturday, March 14, 2015 at 4:43:22 AM UTC-7, Jörg Prante wrote: > > > > > > > > > You may try limit direct memory on JVM level by > > > > > using -XX:MaxDirectMemorySize (default is unlimited). See also > > > > > ES\_DIRECT\_SIZE in [http://www.elastic.co/guide/en/elasticsearch/](http://www.elastic.co/guide/en/elasticsearch/) > > > > > reference/current/setup-service.html#\_linux > > > > > > > > > > I recommend at least 2GB > > > > > > > > > > Jörg > > > > > > > > > > On Sat, Mar 14, 2015 at 1:03 AM, Lindsey Poole [lpo...@gmail.com](mailto:lpo...@gmail.com) > > > > > wrote: > > > > > > > > > > > Hey guys, > > > > > > > > > > > > We're running into some problems under heavy write, nominal read > > > > > > volume when the Lucene memory mapped files have exhausted available > > > > > > physical memory, and segments from disk must be paged into memory. > > > > > > > > > > > > Are there any configs available to control how much physical memory > > > > > > is available to MemoryMapDirectory? > > > > > > > > > > > > Thanks! > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > > msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5% > > > > > > [40googlegroups.com](http://40googlegroups.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/12d0d4ba-8a32-4c85-9635-40d7791865e5%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/cf50ed46-cacf-414b-8b20-b82595dc2fd0%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHY\_M2BAywrG%2BaNcg59xA4\_ocph1oqE0bzba4HTqrdLqQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHY_M2BAywrG%2BaNcg59xA4_ocph1oqE0bzba4HTqrdLqQ%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHY\_M2BAywrG%2BaNcg59xA4\_ocph1oqE0bzba4HTqrdLqQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHY_M2BAywrG%2BaNcg59xA4_ocph1oqE0bzba4HTqrdLqQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4bfeb6fa-5fc4-408c-a888-47553ed00738%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4bfeb6fa-5fc4-408c-a888-47553ed00738%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:26am UTC](https://discuss.elastic.co/t/what-configuration-is-available-to-control-memorymapdirectory/22665/9 "2017-07-06T00:26:40Z")
Closed on July 6, 2017, 12:26am UTC