# Bulk index java not freeing its memory **URL:** \ **Category:** Elasticsearch\ **Created:** [November 27, 2010, 7:49pm UTC](https://discuss.elastic.co/t/bulk-index-java-not-freeing-its-memory/3604 "2010-11-27T19:49:27Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Martijn\_Laarman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/martijn_laarman_2/32/2033_2.png) [@Martijn\_Laarman\_2](https://discuss.elastic.co/u/Martijn_Laarman_2)\ **Post date:** [November 27, 2010, 7:49pm UTC](https://discuss.elastic.co/t/bulk-index-java-not-freeing-its-memory/3604/1 "2010-11-27T19:49:27Z")
I'm creating a rather naive implementation of the bulk API (naive in the sense its not zero copy although i plan too support that later on). Im using the hacker news database dump to insert data en masse into ES see: [https://github.com/Mpdreamz/NEST/blob/master/src/HackerNews.Indexer/Program.cs](https://github.com/Mpdreamz/NEST/blob/master/src/HackerNews.Indexer/Program.cs) This fires a lot of HTTP bulk inserts calls to ES: Example call and response: [https://github.com/Mpdreamz/NEST/blob/master/docs/debug/bulkinsert.txt](https://github.com/Mpdreamz/NEST/blob/master/docs/debug/bulkinsert.txt) If i step through my code after each 'client.IndexAsync(postQueue);' I see java's memory growing but and never recovering from that memory whilst my indexer gets rid of the postQueue in memory. Java jumps from 200mb to 1,5 gb when it starts to spit out javaheap space errors since it can no longer allocate anymore memory. The indexer stays at around 20mb's throughout the process. Any idea what might be going on ? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [November 27, 2010, 9:04pm UTC](https://discuss.elastic.co/t/bulk-index-java-not-freeing-its-memory/3604/2 "2010-11-27T21:04:12Z")
I see that you call IndexAsync, is there a chance that you are simply creating too many concurrent bulk indexing requests into the server? Since you don't wait for the bulk indexing request to get back, there might eventually be hundreds of concurrent bulk indexing requests happening on the server, eventually causing it to max out on mem. If you have 10-15 concurrent indexing actions, does it still happen? On Sat, Nov 27, 2010 at 9:49 PM, Martijn Laarman [mpdreamz@gmail.com](mailto:mpdreamz@gmail.com) wrote: > I'm creating a rather naive implementation of the bulk API (naive in > the sense its not zero copy although i plan too support that later > on). > > Im using the hacker news database dump to insert data en masse into ES > see: > > [https://github.com/Mpdreamz/NEST/blob/master/src/HackerNews.Indexer/Program.cs](https://github.com/Mpdreamz/NEST/blob/master/src/HackerNews.Indexer/Program.cs) > > This fires a lot of HTTP bulk inserts calls to ES: > Example call and response: > [https://github.com/Mpdreamz/NEST/blob/master/docs/debug/bulkinsert.txt](https://github.com/Mpdreamz/NEST/blob/master/docs/debug/bulkinsert.txt) > > If i step through my code after each > 'client.IndexAsync(postQueue);' I see java's memory growing but > and never recovering from that memory whilst my indexer gets rid of > the postQueue in memory. Java jumps from 200mb to 1,5 gb when it > starts to spit out javaheap space errors since it can no longer > allocate anymore memory. The indexer stays at around 20mb's throughout > the process. > > Any idea what might be going on ? ---
**Author:** ![Martijn\_Laarman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/martijn_laarman_2/32/2033_2.png) [@Martijn\_Laarman\_2](https://discuss.elastic.co/u/Martijn_Laarman_2)\ **Post date:** [November 28, 2010, 1:38pm UTC](https://discuss.elastic.co/t/bulk-index-java-not-freeing-its-memory/3604/3 "2010-11-28T13:38:24Z")
thats what i thought at first too but i am putting a breakpoint afterwards and even if i wait for the responses to come back in i can see java's memory growing gradually. Even if i use fiddler to manually fire the requests one by one i am seeing java jumping 4 to 10mb after each request but never releasing it. I am using 13.0 with out of the box settings. Calling \_flush has no effect either. On Sat, Nov 27, 2010 at 10:04 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > I see that you call IndexAsync, is there a chance that you are simply > creating too many concurrent bulk indexing requests into the server? Since > you don't wait for the bulk indexing request to get back, there might > eventually be hundreds of concurrent bulk indexing requests happening on the > server, eventually causing it to max out on mem. If you have 10-15 > concurrent indexing actions, does it still happen? > > On Sat, Nov 27, 2010 at 9:49 PM, Martijn Laarman [mpdreamz@gmail.com](mailto:mpdreamz@gmail.com)wrote: > > > I'm creating a rather naive implementation of the bulk API (naive in > > the sense its not zero copy although i plan too support that later > > on). > > > > Im using the hacker news database dump to insert data en masse into ES > > see: > > > > [https://github.com/Mpdreamz/NEST/blob/master/src/HackerNews.Indexer/Program.cs](https://github.com/Mpdreamz/NEST/blob/master/src/HackerNews.Indexer/Program.cs) > > > > This fires a lot of HTTP bulk inserts calls to ES: > > Example call and response: > > [https://github.com/Mpdreamz/NEST/blob/master/docs/debug/bulkinsert.txt](https://github.com/Mpdreamz/NEST/blob/master/docs/debug/bulkinsert.txt) > > > > If i step through my code after each > > 'client.IndexAsync(postQueue);' I see java's memory growing but > > and never recovering from that memory whilst my indexer gets rid of > > the postQueue in memory. Java jumps from 200mb to 1,5 gb when it > > starts to spit out javaheap space errors since it can no longer > > allocate anymore memory. The indexer stays at around 20mb's throughout > > the process. > > > > Any idea what might be going on ? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [November 28, 2010, 2:15pm UTC](https://discuss.elastic.co/t/bulk-index-java-not-freeing-its-memory/3604/4 "2010-11-28T14:15:11Z")
The JVM will continue to use memory up to what you set it to use (max mem). Once the JVM gets that memory, it will not free it to the OS, but internally, memory will be freed once the garbage collector runs. When you run it one by one, do you still get OutOfMemory errors? On Sun, Nov 28, 2010 at 3:38 PM, Martijn Laarman [mpdreamz@gmail.com](mailto:mpdreamz@gmail.com) wrote: > thats what i thought at first too but i am putting a breakpoint afterwards > and even if i wait for the responses to come back in i can see > java's memory growing gradually. > > Even if i use fiddler to manually fire the requests one by one i am seeing > java jumping 4 to 10mb after each request but never releasing it. > > I am using 13.0 with out of the box settings. > > Calling \_flush has no effect either. > > On Sat, Nov 27, 2010 at 10:04 PM, Shay Banon \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > > > wrote: > > > I see that you call IndexAsync, is there a chance that you are simply > > creating too many concurrent bulk indexing requests into the server? Since > > you don't wait for the bulk indexing request to get back, there might > > eventually be hundreds of concurrent bulk indexing requests happening on the > > server, eventually causing it to max out on mem. If you have 10-15 > > concurrent indexing actions, does it still happen? > > > > On Sat, Nov 27, 2010 at 9:49 PM, Martijn Laarman [mpdreamz@gmail.com](mailto:mpdreamz@gmail.com)wrote: > > > > > I'm creating a rather naive implementation of the bulk API (naive in > > > the sense its not zero copy although i plan too support that later > > > on). > > > > > > Im using the hacker news database dump to insert data en masse into ES > > > see: > > > > > > [https://github.com/Mpdreamz/NEST/blob/master/src/HackerNews.Indexer/Program.cs](https://github.com/Mpdreamz/NEST/blob/master/src/HackerNews.Indexer/Program.cs) > > > > > > This fires a lot of HTTP bulk inserts calls to ES: > > > Example call and response: > > > [https://github.com/Mpdreamz/NEST/blob/master/docs/debug/bulkinsert.txt](https://github.com/Mpdreamz/NEST/blob/master/docs/debug/bulkinsert.txt) > > > > > > If i step through my code after each > > > 'client.IndexAsync(postQueue);' I see java's memory growing but > > > and never recovering from that memory whilst my indexer gets rid of > > > the postQueue in memory. Java jumps from 200mb to 1,5 gb when it > > > starts to spit out javaheap space errors since it can no longer > > > allocate anymore memory. The indexer stays at around 20mb's throughout > > > the process. > > > > > > Any idea what might be going on ? ---
**Author:** ![Martijn\_Laarman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/martijn_laarman_2/32/2033_2.png) [@Martijn\_Laarman\_2](https://discuss.elastic.co/u/Martijn_Laarman_2)\ **Post date:** [November 28, 2010, 10:30pm UTC](https://discuss.elastic.co/t/bulk-index-java-not-freeing-its-memory/3604/5 "2010-11-28T22:30:24Z")
Ahh i saw java eating mem and assumed the problem was there. NEST now has built in semaphore support for async connections which solved the issues. Setting the maxasyncconnection too high still throws outofmem exceptions but its easy to tweak now. Thanks 🙂 On Sun, Nov 28, 2010 at 3:15 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > The JVM will continue to use memory up to what you set it to use (max mem). > Once the JVM gets that memory, it will not free it to the OS, but > internally, memory will be freed once the garbage collector runs. > > When you run it one by one, do you still get OutOfMemory errors? > > On Sun, Nov 28, 2010 at 3:38 PM, Martijn Laarman [mpdreamz@gmail.com](mailto:mpdreamz@gmail.com)wrote: > > > thats what i thought at first too but i am putting a breakpoint afterwards > > and even if i wait for the responses to come back in i can see > > java's memory growing gradually. > > > > Even if i use fiddler to manually fire the requests one by one i am seeing > > java jumping 4 to 10mb after each request but never releasing it. > > > > I am using 13.0 with out of the box settings. > > > > Calling \_flush has no effect either. > > > > On Sat, Nov 27, 2010 at 10:04 PM, Shay Banon \< > > [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)\> wrote: > > > > > I see that you call IndexAsync, is there a chance that you are simply > > > creating too many concurrent bulk indexing requests into the server? Since > > > you don't wait for the bulk indexing request to get back, there might > > > eventually be hundreds of concurrent bulk indexing requests happening on the > > > server, eventually causing it to max out on mem. If you have 10-15 > > > concurrent indexing actions, does it still happen? > > > > > > On Sat, Nov 27, 2010 at 9:49 PM, Martijn Laarman [mpdreamz@gmail.com](mailto:mpdreamz@gmail.com)wrote: > > > > > > > I'm creating a rather naive implementation of the bulk API (naive in > > > > the sense its not zero copy although i plan too support that later > > > > on). > > > > > > > > Im using the hacker news database dump to insert data en masse into ES > > > > see: > > > > > > > > [https://github.com/Mpdreamz/NEST/blob/master/src/HackerNews.Indexer/Program.cs](https://github.com/Mpdreamz/NEST/blob/master/src/HackerNews.Indexer/Program.cs) > > > > > > > > This fires a lot of HTTP bulk inserts calls to ES: > > > > Example call and response: > > > > [https://github.com/Mpdreamz/NEST/blob/master/docs/debug/bulkinsert.txt](https://github.com/Mpdreamz/NEST/blob/master/docs/debug/bulkinsert.txt) > > > > > > > > If i step through my code after each > > > > 'client.IndexAsync(postQueue);' I see java's memory growing but > > > > and never recovering from that memory whilst my indexer gets rid of > > > > the postQueue in memory. Java jumps from 200mb to 1,5 gb when it > > > > starts to spit out javaheap space errors since it can no longer > > > > allocate anymore memory. The indexer stays at around 20mb's throughout > > > > the process. > > > > > > > > Any idea what might be going on ? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:16am UTC](https://discuss.elastic.co/t/bulk-index-java-not-freeing-its-memory/3604/6 "2017-07-06T04:16:03Z")