# Java.lang.OutOfMemoryError: Java heap space **URL:** \ **Category:** Elasticsearch\ **Created:** [June 18, 2013, 5:52pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466 "2013-06-18T17:52:12Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![vinod\_eligeti](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_eligeti](https://discuss.elastic.co/u/vinod_eligeti)\ **Post date:** [June 18, 2013, 5:52pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/1 "2013-06-18T17:52:12Z")
I am getting heap space error. Basically I am starting a embedded ES node with data set to true option. The elasticsearch.yml is not in the classpath which means it will go for defaults. I created 1 index seeded 3 docs and the size of each doc is not more than 1000 bytes. And when I ran this query query: { "size" : 2147483646, "query" : { "bool" : { "should" : { "term" : { "account-name" : "customer1" } }, "minimum\_should\_match" : "1" } }, "filter" : { "range" : { "timestamp" : { "from" : 1371577346230, "to" : 1371577406229, "include\_lower" : true, "include\_upper" : true } } } } And then i get this: Exception in thread "elasticsearch[Hulk][search][T#1]" java.lang.OutOfMemoryError: Java heap space at org.elasticsearch.search.SearchService.shortcutDocIdsToLoad(SearchService.java:588) at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:325) at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:243) at org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryAndFetchAction.java:75) at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205) at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192) at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:178) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) Why such a small data I am getting Java heap error. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![vinod\_eligeti](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_eligeti](https://discuss.elastic.co/u/vinod_eligeti)\ **Post date:** [June 18, 2013, 5:54pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/2 "2013-06-18T17:54:31Z")
The node I created has these settings NodeBuilder nb = new NodeBuilder().clusterName("my-store"); nb.settings().put("node.data", true); nb.settings().put("index.number\_of\_shards", 1); nb.settings().put("index.number\_of\_replicas", 0); nb.settings().put("ES\_MIN\_MEM", "2g"); nb.settings().put("ES\_MAX\_MEM", "2g"); Node node = nb.client(false).node(); On Tuesday, June 18, 2013 10:52:12 AM UTC-7, vinod eligeti wrote: > I am getting heap space error. > > Basically I am starting a embedded ES node with data set to true option. > The elasticsearch.yml is not in the classpath which means it will go for > defaults. I created 1 index seeded 3 docs and the size of each doc is not > more than 1000 bytes. And when I ran this query > > query: { > "size" : 2147483646, > "query" : { > "bool" : { > "should" : { > "term" : { > "account-name" : "customer1" > } > }, > "minimum\_should\_match" : "1" > } > }, > "filter" : { > "range" : { > "timestamp" : { > "from" : 1371577346230, > "to" : 1371577406229, > "include\_lower" : true, > "include\_upper" : true > } > } > } > } > > And then i get this: > > Exception in thread "elasticsearch[Hulk][search][T#1]" > java.lang.OutOfMemoryError: Java heap space > at > org.elasticsearch.search.SearchService.shortcutDocIdsToLoad(SearchService.java:588) > at > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:325) > at > org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:243) > at > org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryAndFetchAction.java:75) > at > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205) > at > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192) > at > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:178) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:680) > > Why such a small data I am getting Java heap error. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 18, 2013, 7:24pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/3 "2013-06-18T19:24:29Z")
It won't work. Your JVM heap size must be set outside your program. Typically, set JAVA\_OPTS=-Xmx2g -Xms2g or something like that. Then run your program. The JVM will allocate memory for it. Embedded ES nodes will use it. Make sense? -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 18 juin 2013 à 19:54, vinod eligeti [veligeti999@gmail.com](mailto:veligeti999@gmail.com) a écrit : The node I created has these settings ``` NodeBuilder nb = new NodeBuilder().clusterName("my-store"); nb.settings().put("node.data", true); nb.settings().put("index.number_of_shards", 1); nb.settings().put("index.number_of_replicas", 0); nb.settings().put("ES_MIN_MEM", "2g"); nb.settings().put("ES_MAX_MEM", "2g"); Node node = nb.client(false).node(); ``` On Tuesday, June 18, 2013 10:52:12 AM UTC-7, vinod eligeti wrote: > I am getting heap space error. > > Basically I am starting a embedded ES node with data set to true option. The elasticsearch.yml is not in the classpath which means it will go for defaults. I created 1 index seeded 3 docs and the size of each doc is not more than 1000 bytes. And when I ran this query > > query: { > "size" : 2147483646, > "query" : { > "bool" : { > "should" : { > "term" : { > "account-name" : "customer1" > } > }, > "minimum\_should\_match" : "1" > } > }, > "filter" : { > "range" : { > "timestamp" : { > "from" : 1371577346230, > "to" : 1371577406229, > "include\_lower" : true, > "include\_upper" : true > } > } > } > } > > And then i get this: > > Exception in thread "elasticsearch[Hulk][search][T#1]" java.lang.OutOfMemoryError: Java heap space > at org.elasticsearch.search.SearchService.shortcutDocIdsToLoad(SearchService.java:588) > at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:325) > at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:243) > at org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryAndFetchAction.java:75) > at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205) > at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192) > at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:178) > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:680) > > Why such a small data I am getting Java heap error. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [June 19, 2013, 7:40pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/4 "2013-06-19T19:40:20Z")
And why are you requesting size 2147483646??? On 18 June 2013 21:24, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > It won't work. > Your JVM heap size must be set outside your program. > > Typically, set JAVA\_OPTS=-Xmx2g -Xms2g or something like that. > Then run your program. The JVM will allocate memory for it. > Embedded ES nodes will use it. > > Make sense? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 18 juin 2013 à 19:54, vinod eligeti [veligeti999@gmail.com](mailto:veligeti999@gmail.com) a écrit : > > The node I created has these settings > > NodeBuilder nb = new NodeBuilder().clusterName("my-store"); > nb.settings().put("node.data", true); > nb.settings().put("index.number\_of\_shards", 1); > nb.settings().put("index.number\_of\_replicas", 0); > nb.settings().put("ES\_MIN\_MEM", "2g"); > nb.settings().put("ES\_MAX\_MEM", "2g"); > > Node node = nb.client(false).node(); > > On Tuesday, June 18, 2013 10:52:12 AM UTC-7, vinod eligeti wrote: > > > I am getting heap space error. > > > > Basically I am starting a embedded ES node with data set to true option. > > The elasticsearch.yml is not in the classpath which means it will go for > > defaults. I created 1 index seeded 3 docs and the size of each doc is not > > more than 1000 bytes. And when I ran this query > > > > query: { > > "size" : 2147483646, > > "query" : { > > "bool" : { > > "should" : { > > "term" : { > > "account-name" : "customer1" > > } > > }, > > "minimum\_should\_match" : "1" > > } > > }, > > "filter" : { > > "range" : { > > "timestamp" : { > > "from" : 1371577346230, > > "to" : 1371577406229, > > "include\_lower" : true, > > "include\_upper" : true > > } > > } > > } > > } > > > > And then i get this: > > > > Exception in thread "elasticsearch[Hulk][search][\*\*T#1]" > > java.lang.OutOfMemoryError: Java heap space > > at org.elasticsearch.search.\*\*SearchService.**shortcutDocIdsToLoad(** > > SearchService.java:588) > > at org.elasticsearch.search.\*\*SearchService.**executeFetchPhase(** > > SearchService.java:325) > > at org.elasticsearch.search.\*\*action. **SearchServiceTransportAction.** > > sendExecuteFetch(\*\*SearchServiceTransportAction.**java:243) > > at org.elasticsearch.action.search.type. > > TransportSearchQueryAndFetchAc**tion$AsyncAction.\*\*sendExecuteFirstPhase( > > **TransportSearchQueryAndFetchAc** tion.java:75) > > at org.elasticsearch.action.\*\*search.type. **TransportSearchTypeAction$** > > BaseAsyncAction.\*\*performFirstPhase( **TransportSearchTypeAction.** > > java:205) > > at org.elasticsearch.action.\*\*search.type. **TransportSearchTypeAction$** > > BaseAsyncAction.\*\*performFirstPhase( **TransportSearchTypeAction.** > > java:192) > > at org.elasticsearch.action.\*\*search.type. **TransportSearchTypeAction$** > > BaseAsyncAction$2.run(\*\*TransportSearchTypeAction.\*\*java:178) > > at java.util.concurrent. **ThreadPoolExecutor$Worker.** > > runTask(ThreadPoolExecutor.\*\*java:886) > > at java.util.concurrent.**ThreadPoolExecutor$Worker.run(** > > ThreadPoolExecutor.java:908) > > at java.lang.Thread.run(Thread.\*\*java:680) > > > > Why such a small data I am getting Java heap error. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![vinod\_eligeti](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_eligeti](https://discuss.elastic.co/u/vinod_eligeti)\ **Post date:** [June 19, 2013, 7:58pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/5 "2013-06-19T19:58:58Z")
Yes and I figured that is the problem. Sorry not to update soon. On Wed, Jun 19, 2013 at 12:40 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > And why are you requesting size 2147483646??? > > On 18 June 2013 21:24, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > > > It won't work. > > Your JVM heap size must be set outside your program. > > > > Typically, set JAVA\_OPTS=-Xmx2g -Xms2g or something like that. > > Then run your program. The JVM will allocate memory for it. > > Embedded ES nodes will use it. > > > > Make sense? > > > > -- > > David 😉 > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > Le 18 juin 2013 à 19:54, vinod eligeti [veligeti999@gmail.com](mailto:veligeti999@gmail.com) a écrit : > > > > The node I created has these settings > > > > NodeBuilder nb = new NodeBuilder().clusterName("my-store"); > > nb.settings().put("node.data", true); > > nb.settings().put("index.number\_of\_shards", 1); > > nb.settings().put("index.number\_of\_replicas", 0); > > nb.settings().put("ES\_MIN\_MEM", "2g"); > > nb.settings().put("ES\_MAX\_MEM", "2g"); > > > > Node node = nb.client(false).node(); > > > > On Tuesday, June 18, 2013 10:52:12 AM UTC-7, vinod eligeti wrote: > > > > > I am getting heap space error. > > > > > > Basically I am starting a embedded ES node with data set to true option. > > > The elasticsearch.yml is not in the classpath which means it will go for > > > defaults. I created 1 index seeded 3 docs and the size of each doc is not > > > more than 1000 bytes. And when I ran this query > > > > > > query: { > > > "size" : 2147483646, > > > "query" : { > > > "bool" : { > > > "should" : { > > > "term" : { > > > "account-name" : "customer1" > > > } > > > }, > > > "minimum\_should\_match" : "1" > > > } > > > }, > > > "filter" : { > > > "range" : { > > > "timestamp" : { > > > "from" : 1371577346230, > > > "to" : 1371577406229, > > > "include\_lower" : true, > > > "include\_upper" : true > > > } > > > } > > > } > > > } > > > > > > And then i get this: > > > > > > Exception in thread "elasticsearch[Hulk][search][\*\*T#1]" > > > java.lang.OutOfMemoryError: Java heap space > > > at org.elasticsearch.search.\*\*SearchService.**shortcutDocIdsToLoad(** > > > SearchService.java:588) > > > at org.elasticsearch.search.\*\*SearchService.**executeFetchPhase(** > > > SearchService.java:325) > > > at org.elasticsearch.search.\*\*action. **SearchServiceTransportAction.** > > > sendExecuteFetch(**SearchServiceTransportAction.java:243) > > > at org.elasticsearch.action.search.type. > > > TransportSearchQueryAndFetchAction$AsyncAction.** > > > sendExecuteFirstPhase( **TransportSearchQueryAndFetchAc** tion.java:75) > > > at org.elasticsearch.action.\*\*search.type.\*_TransportSearchTypeAction$_ > > > \*BaseAsyncAction.\*\*performFirstPhase( **TransportSearchTypeAction.** > > > java:205) > > > at org.elasticsearch.action.\*\*search.type.\*_TransportSearchTypeAction$_ > > > \*BaseAsyncAction.\*\*performFirstPhase( **TransportSearchTypeAction.** > > > java:192) > > > at org.elasticsearch.action.\*\*search.type.\*_TransportSearchTypeAction$_ > > > \*BaseAsyncAction$2.run(\*\*TransportSearchTypeAction.\*\*java:178) > > > at java.util.concurrent. **ThreadPoolExecutor$Worker.** > > > runTask(ThreadPoolExecutor.\*\*java:886) > > > at java.util.concurrent.**ThreadPoolExecutor$Worker.run(** > > > ThreadPoolExecutor.java:908) > > > at java.lang.Thread.run(Thread.\*\*java:680) > > > > > > Why such a small data I am getting Java heap error. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 19, 2013, 9:34pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/6 "2013-06-19T21:34:22Z")
Oh my god! I completely missed that!!! Thanks Clint! -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 19 juin 2013 à 21:40, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com) a écrit : And why are you requesting size 2147483646??? On 18 June 2013 21:24, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > It won't work. > Your JVM heap size must be set outside your program. > > Typically, set JAVA\_OPTS=-Xmx2g -Xms2g or something like that. > Then run your program. The JVM will allocate memory for it. > Embedded ES nodes will use it. > > Make sense? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 18 juin 2013 à 19:54, vinod eligeti [veligeti999@gmail.com](mailto:veligeti999@gmail.com) a écrit : > > The node I created has these settings > > ``` > NodeBuilder nb = new NodeBuilder().clusterName("my-store"); > nb.settings().put("node.data", true); > nb.settings().put("index.number_of_shards", 1); > nb.settings().put("index.number_of_replicas", 0); > nb.settings().put("ES_MIN_MEM", "2g"); > nb.settings().put("ES_MAX_MEM", "2g"); > > Node node = nb.client(false).node(); > > ``` > > On Tuesday, June 18, 2013 10:52:12 AM UTC-7, vinod eligeti wrote: > > > I am getting heap space error. > > > > Basically I am starting a embedded ES node with data set to true option. The elasticsearch.yml is not in the classpath which means it will go for defaults. I created 1 index seeded 3 docs and the size of each doc is not more than 1000 bytes. And when I ran this query > > > > query: { > > "size" : 2147483646, > > "query" : { > > "bool" : { > > "should" : { > > "term" : { > > "account-name" : "customer1" > > } > > }, > > "minimum\_should\_match" : "1" > > } > > }, > > "filter" : { > > "range" : { > > "timestamp" : { > > "from" : 1371577346230, > > "to" : 1371577406229, > > "include\_lower" : true, > > "include\_upper" : true > > } > > } > > } > > } > > > > And then i get this: > > > > Exception in thread "elasticsearch[Hulk][search][T#1]" java.lang.OutOfMemoryError: Java heap space > > at org.elasticsearch.search.SearchService.shortcutDocIdsToLoad(SearchService.java:588) > > at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:325) > > at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:243) > > at org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryAndFetchAction.java:75) > > at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205) > > at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192) > > at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:178) > > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > > at java.lang.Thread.run(Thread.java:680) > > > > Why such a small data I am getting Java heap error. > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![James\_McGill](https://avatars.discourse-cdn.com/v4/letter/j/8797f3/32.png) [@James\_McGill](https://discuss.elastic.co/u/James_McGill)\ **Post date:** [July 21, 2013, 4:29pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/7 "2013-07-21T16:29:04Z")
When you request an extremely large size query result (in your case, apparently Integer.MAX\_VALUE -1) Elasticsearch will attempt to allocate an array of that size. You might have a look at the source line where the OOM occurs, and you will see it, an allocation of int[context.size()], the equivalent of new int[Integer.MAX\_VALUE -1]. So the idiom of setting a huge query size in order to "select everything" won't work for Elasticsearch. You either need to know the max size of your query (perhaps doing a count query first) or use the scan/scroll API. On Tuesday, June 18, 2013 10:52:12 AM UTC-7, vinod eligeti wrote: > I am getting heap space error. > > Basically I am starting a embedded ES node with data set to true option. > The elasticsearch.yml is not in the classpath which means it will go for > defaults. I created 1 index seeded 3 docs and the size of each doc is not > more than 1000 bytes. And when I ran this query > > query: { > "size" : 2147483646, > "query" : { > "bool" : { > "should" : { > "term" : { > "account-name" : "customer1" > } > }, > "minimum\_should\_match" : "1" > } > }, > "filter" : { > "range" : { > "timestamp" : { > "from" : 1371577346230, > "to" : 1371577406229, > "include\_lower" : true, > "include\_upper" : true > } > } > } > } > > And then i get this: > > Exception in thread "elasticsearch[Hulk][search][T#1]" > java.lang.OutOfMemoryError: Java heap space > at > org.elasticsearch.search.SearchService.shortcutDocIdsToLoad(SearchService.java:588) > at > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:325) > at > org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:243) > at > org.elasticsearch.action.search.type.TransportSearchQueryAndFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryAndFetchAction.java:75) > at > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205) > at > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192) > at > org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:178) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:680) > > Why such a small data I am getting Java heap error. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Massimiliano\_Peranto](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@Massimiliano\_Peranto](https://discuss.elastic.co/u/Massimiliano_Peranto)\ **Post date:** [October 3, 2013, 5:03am UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/8 "2013-10-03T05:03:28Z")
Hi, I have the same problem. I got a single node which has a tomcat frontend where I just created a servlet to convert a specific frontend query language to ES JSON queries. Everything works quite fine, until I use JMeter to test performance of the system. At that point I get, after several queries, an OutOfMemoryError. I tried increasing memory heap up to 2g but nothing. The index on disk, actually, takes less than a GB, so what's happening? My queries deal with filters and query strings, no facets at the moment. I double checked that all the suggestions seen in this post were applied (size of result = 30, 2g of heap). At the moment as I increase the memory for higher values I get an aborted, due to memory limits per process but, anyway, the index is 900Mb! Why would it occupy in memory more than this? Just for testing, is there any kind of query caching that may be disabled? The aim of the project is to index hundreds of GBs of emails... Actually we got not more than a 1000 mails for the indicated index size... Can you help me? Thanks! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 3, 2013, 5:32am UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/9 "2013-10-03T05:32:36Z")
If you use JMeter, that means that you are testing your PROD platform? And you won't give more than 2Gb heap? How many concurrent requests did you send? -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 3 oct. 2013 à 07:03, Massimiliano Perantoni [massimiliano.perantoni@gmail.com](mailto:massimiliano.perantoni@gmail.com) a écrit : > Hi, > I have the same problem. I got a single node which has a tomcat frontend where I just created a servlet to convert a specific frontend query language to ES JSON queries. > Everything works quite fine, until I use JMeter to test performance of the system. At that point I get, after several queries, an OutOfMemoryError. > I tried increasing memory heap up to 2g but nothing. The index on disk, actually, takes less than a GB, so what's happening? > My queries deal with filters and query strings, no facets at the moment. > > I double checked that all the suggestions seen in this post were applied (size of result = 30, 2g of heap). At the moment as I increase the memory for higher values I get an aborted, due to memory limits per process but, anyway, the index is 900Mb! Why would it occupy in memory more than this? Just for testing, is there any kind of query caching that may be disabled? > The aim of the project is to index hundreds of GBs of emails... Actually we got not more than a 1000 mails for the indicated index size... > > ## Can you help me? Thanks! > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Massimiliano\_Peranto](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@Massimiliano\_Peranto](https://discuss.elastic.co/u/Massimiliano_Peranto)\ **Post date:** [October 3, 2013, 7:01am UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/10 "2013-10-03T07:01:03Z")
Well, I'd give it even more, but as soon as I reach 4G of heap the process get aborted... That's actually a POC platform, but we're sizing the PROD system on it. I have to understand if it's a "hardware" limit or if it's a leak... Up til now I noticed that as I grow it takes. I don't actually know how ES works in allocating memory, but I'd expect that memorìy usage it should not overtake the index size... Is it that wrong as an assuption? java -version java version "1.7.0\_21" uname -a Linux mercury 3.10.10-200.fc19.x86\_64 #1 SMP Thu Aug 29 19:05:45 UTC 2013 x86\_64 x86\_64 x86\_64 GNU/Linux ulimit -u unlimited ulimit -l unlimited everything, to do a first test, is running as root. Thanks for any help! 2013/10/3 David Pilato [david@pilato.fr](mailto:david@pilato.fr): > If you use JMeter, that means that you are testing your PROD platform? > And you won't give more than 2Gb heap? > > How many concurrent requests did you send? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 3 oct. 2013 à 07:03, Massimiliano Perantoni > [massimiliano.perantoni@gmail.com](mailto:massimiliano.perantoni@gmail.com) a écrit : > > Hi, > I have the same problem. I got a single node which has a tomcat frontend > where I just created a servlet to convert a specific frontend query language > to ES JSON queries. > Everything works quite fine, until I use JMeter to test performance of the > system. At that point I get, after several queries, an OutOfMemoryError. > I tried increasing memory heap up to 2g but nothing. The index on disk, > actually, takes less than a GB, so what's happening? > My queries deal with filters and query strings, no facets at the moment. > > I double checked that all the suggestions seen in this post were applied > (size of result = 30, 2g of heap). At the moment as I increase the memory > for higher values I get an aborted, due to memory limits per process but, > anyway, the index is 900Mb! Why would it occupy in memory more than this? > Just for testing, is there any kind of query caching that may be disabled? > The aim of the project is to index hundreds of GBs of emails... Actually we > got not more than a 1000 mails for the indicated index size... > > Can you help me? > Thanks! > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Massimiliano\_Peranto](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@Massimiliano\_Peranto](https://discuss.elastic.co/u/Massimiliano_Peranto)\ **Post date:** [October 3, 2013, 7:16am UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/11 "2013-10-03T07:16:23Z")
Just to be complete, the concurrency just delays or anticipates the Error... If I use just 1 connection or 20 connections I get the "system death" earlier or later. As the web and the ES engine live on the same system, actually I had to reduce the number of connections to ES, as concurrency brought the system to socket starvation. One more thing... I'm using this way to connect... Client client = new TransportClient() .addTransportAddress(new InetSocketTransportAddress( "127.0.0.1", 9300)); [.... doing my query...] client.close(); everything in the servlet. As soon as I close the client, I'd expect that when the method using it completes the GC cleans it (some day, not immediately), but that the close() method closes the socket... Always using JMeter we noticed that this is not true... We get a socket starvation after some minutes of testing. After that I started using a connectionpool, with 20 connections, and everything is working OK, but now we get the OOME.... Thanks! 2013/10/3 David Pilato [david@pilato.fr](mailto:david@pilato.fr): > If you use JMeter, that means that you are testing your PROD platform? > And you won't give more than 2Gb heap? > > How many concurrent requests did you send? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 3 oct. 2013 à 07:03, Massimiliano Perantoni > [massimiliano.perantoni@gmail.com](mailto:massimiliano.perantoni@gmail.com) a écrit : > > Hi, > I have the same problem. I got a single node which has a tomcat frontend > where I just created a servlet to convert a specific frontend query language > to ES JSON queries. > Everything works quite fine, until I use JMeter to test performance of the > system. At that point I get, after several queries, an OutOfMemoryError. > I tried increasing memory heap up to 2g but nothing. The index on disk, > actually, takes less than a GB, so what's happening? > My queries deal with filters and query strings, no facets at the moment. > > I double checked that all the suggestions seen in this post were applied > (size of result = 30, 2g of heap). At the moment as I increase the memory > for higher values I get an aborted, due to memory limits per process but, > anyway, the index is 900Mb! Why would it occupy in memory more than this? > Just for testing, is there any kind of query caching that may be disabled? > The aim of the project is to index hundreds of GBs of emails... Actually we > got not more than a 1000 mails for the indicated index size... > > Can you help me? > Thanks! > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [October 3, 2013, 9:58am UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/12 "2013-10-03T09:58:32Z")
You assumption the index file size on disk should match maximum heap is wrong. First, the index is bigger than on disk, since compression is used. It may be a factor of 5 or 10 or even more. Second, more important, the index never loads on the heap, but into the filesystem cache. If you observe OOM there is something wrong with the query. Can you please show the queries you use, and the full stack trace? Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 3, 2013, 10:54am UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/13 "2013-10-03T10:54:24Z")
In addition to this, make sure that you have only one TransportClient for the full JVM. You don't need and you should not create many clients. -- David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) @dadoonet | @elasticsearchfr | @scrutmydocs Le 3 oct. 2013 à 11:58, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) a écrit : > You assumption the index file size on disk should match maximum heap is wrong. > > First, the index is bigger than on disk, since compression is used. It may be a factor of 5 or 10 or even more. > > Second, more important, the index never loads on the heap, but into the filesystem cache. > > If you observe OOM there is something wrong with the query. > > Can you please show the queries you use, and the full stack trace? > > Jörg > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Massimiliano\_Peranto](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@Massimiliano\_Peranto](https://discuss.elastic.co/u/Massimiliano_Peranto)\ **Post date:** [October 3, 2013, 2:24pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/14 "2013-10-03T14:24:00Z")
Hi, dealing with Client number the problem doesn't change that much. If I just connect once (I create a pool with just one connection), I'll get the same problem, after some time the system is running. Just to be sure that I understood... Should I call the close() method each time I end a query? Or I should close it just when I close the client application? dealing with queries, these are various queries that I do, that would be a big log... Can I send it? Which other information may be useful to you? Ciao! 2013/10/3 David Pilato [david@pilato.fr](mailto:david@pilato.fr): > In addition to this, make sure that you have only one TransportClient for > the full JVM. You don't need and you should not create many clients. > > -- > David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) > @dadoonet | @elasticsearchfr | @scrutmydocs > > Le 3 oct. 2013 à 11:58, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) a > écrit : > > You assumption the index file size on disk should match maximum heap is > wrong. > > First, the index is bigger than on disk, since compression is used. It may > be a factor of 5 or 10 or even more. > > Second, more important, the index never loads on the heap, but into the > filesystem cache. > > If you observe OOM there is something wrong with the query. > > Can you please show the queries you use, and the full stack trace? > > Jörg > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 3, 2013, 2:55pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/15 "2013-10-03T14:55:20Z")
Only call close when you close the client app. -- David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) @dadoonet | @elasticsearchfr | @scrutmydocs Le 3 oct. 2013 à 16:24, Massimiliano Perantoni [massimiliano.perantoni@gmail.com](mailto:massimiliano.perantoni@gmail.com) a écrit : > Hi, > dealing with Client number the problem doesn't change that much. If I > just connect once (I create a pool with just one connection), I'll get > the same problem, after some time the system is running. > Just to be sure that I understood... Should I call the close() method > each time I end a query? Or I should close it just when I close the > client application? > dealing with queries, these are various queries that I do, that would > be a big log... Can I send it? Which other information may be useful > to you? > Ciao! > > 2013/10/3 David Pilato [david@pilato.fr](mailto:david@pilato.fr): > > > In addition to this, make sure that you have only one TransportClient for > > the full JVM. You don't need and you should not create many clients. > > > > -- > > David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) > > @dadoonet | @elasticsearchfr | @scrutmydocs > > > > Le 3 oct. 2013 à 11:58, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) a > > écrit : > > > > You assumption the index file size on disk should match maximum heap is > > wrong. > > > > First, the index is bigger than on disk, since compression is used. It may > > be a factor of 5 or 10 or even more. > > > > Second, more important, the index never loads on the heap, but into the > > filesystem cache. > > > > If you observe OOM there is something wrong with the query. > > > > Can you please show the queries you use, and the full stack trace? > > > > Jörg > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![James\_Richardson](https://avatars.discourse-cdn.com/v4/letter/j/838e76/32.png) [@James\_Richardson](https://discuss.elastic.co/u/James_Richardson)\ **Post date:** [October 3, 2013, 3:44pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/16 "2013-10-03T15:44:28Z")
For what it's worth, i am searching 1m documents, about 5G, using a max heap of 350m and its all working just fine. This includes facets over the entire index... -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Massimiliano\_Peranto](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@Massimiliano\_Peranto](https://discuss.elastic.co/u/Massimiliano_Peranto)\ **Post date:** [October 3, 2013, 3:47pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/17 "2013-10-03T15:47:06Z")
Have you ever stressed the engine to test its reactions? 2013/10/3 James Richardson [james.time4tea@gmail.com](mailto:james.time4tea@gmail.com): > For what it's worth, i am searching 1m documents, about 5G, using a max heap > of 350m and its all working just fine. This includes facets over the entire > index... > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![James\_Richardson\_2](https://avatars.discourse-cdn.com/v4/letter/j/97f17d/32.png) [@James\_Richardson\_2](https://discuss.elastic.co/u/James_Richardson_2)\ **Post date:** [October 3, 2013, 3:52pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/18 "2013-10-03T15:52:23Z")
Well, I have tested that it works with the load I need. I don't really see the point in creating some imaginary load levels and just stressing it out. If it works for what I needs that's good enough... The more important thing (for me in this instance) is that it works _enough_ and the heap size is small enough that it can run on a cheap vm. James On 3 October 2013 16:47, Massimiliano Perantoni \< [massimiliano.perantoni@gmail.com](mailto:massimiliano.perantoni@gmail.com)\> wrote: > Have you ever stressed the engine to test its reactions? > > 2013/10/3 James Richardson [james.time4tea@gmail.com](mailto:james.time4tea@gmail.com): > > > For what it's worth, i am searching 1m documents, about 5G, using a max > > heap > > of 350m and its all working just fine. This includes facets over the > > entire > > index... > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Massimiliano\_Peranto](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@Massimiliano\_Peranto](https://discuss.elastic.co/u/Massimiliano_Peranto)\ **Post date:** [October 3, 2013, 3:56pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/19 "2013-10-03T15:56:50Z")
Actually we'll have to stand the load of hundreds of simultaneous queries... 2013/10/3 James Richardson [james@time4tea.net](mailto:james@time4tea.net): > Well, I have tested that it works with the load I need. I don't really see > the point in creating some imaginary load levels and just stressing it out. > If it works for what I needs that's good enough... The more important thing > (for me in this instance) is that it works _enough_ and the heap size is > small enough that it can run on a cheap vm. > > James > > On 3 October 2013 16:47, Massimiliano Perantoni > [massimiliano.perantoni@gmail.com](mailto:massimiliano.perantoni@gmail.com) wrote: > > > Have you ever stressed the engine to test its reactions? > > > > 2013/10/3 James Richardson [james.time4tea@gmail.com](mailto:james.time4tea@gmail.com): > > > > > For what it's worth, i am searching 1m documents, about 5G, using a max > > > heap > > > of 350m and its all working just fine. This includes facets over the > > > entire > > > index... > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![James\_Richardson\_2](https://avatars.discourse-cdn.com/v4/letter/j/97f17d/32.png) [@James\_Richardson\_2](https://discuss.elastic.co/u/James_Richardson_2)\ **Post date:** [October 3, 2013, 4:35pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466/20 "2013-10-03T16:35:35Z")
I quite often find that the real number of actually simultaneous things is often fewer than you might first think - each query only lasts a few ms, often the results can be cached using squid or varnish, lots of times people are searching common content for the same stuff, each of which reduces the number of simultaneous queries that you have to do. In order to support 100's of users, you might find that really only 1 or 2 queries are actually happening at the same time... I mean YMMV... On 3 October 2013 16:56, Massimiliano Perantoni \< [massimiliano.perantoni@gmail.com](mailto:massimiliano.perantoni@gmail.com)\> wrote: > Actually we'll have to stand the load of hundreds of simultaneous > queries... > > 2013/10/3 James Richardson [james@time4tea.net](mailto:james@time4tea.net): > > > Well, I have tested that it works with the load I need. I don't really > > see > > the point in creating some imaginary load levels and just stressing it > > out. > > If it works for what I needs that's good enough... The more important > > thing > > (for me in this instance) is that it works _enough_ and the heap size is > > small enough that it can run on a cheap vm. > > > > James > > > > On 3 October 2013 16:47, Massimiliano Perantoni > > [massimiliano.perantoni@gmail.com](mailto:massimiliano.perantoni@gmail.com) wrote: > > > > > Have you ever stressed the engine to test its reactions? > > > > > > 2013/10/3 James Richardson [james.time4tea@gmail.com](mailto:james.time4tea@gmail.com): > > > > > > > For what it's worth, i am searching 1m documents, about 5G, using a > > > > max > > > > heap > > > > of 350m and its all working just fine. This includes facets over the > > > > entire > > > > index... > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe) > > > . > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/CfQsLQWqGRg/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). [Next page](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/12466.md?page=2)