# OutOfMemory using bulk insert and a refresh interval of -1
**URL:** https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458
**Category:** Elasticsearch
**Created:** [June 10, 2016, 3:31pm UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458 "2016-06-10T15:31:28Z")
**Posts on this page:** 19
**Page:** 1
### Author:  [@franekrichardson](https://discuss.elastic.co/u/franekrichardson)
#### Post date: [June 10, 2016, 3:31pm UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/1 "2016-06-10T15:31:28Z")
Hi,
I'm using the scala ES library elastic4s which is just a wrapper around the java client. And I've come across what feels like a bug in some local testing, I've been loading an index with ~30 million documents (each quite small a few hundred bytes at most). To do this I've been bulk inserting with a bulk size of 1000, I then set the refresh interval to -1 to try to improve the performance, I found the Es instance would fairly rapidly fail with OOM.
Watching it in visualvm you can see the GC kicks in but recovers progressively less as it goes, whereas with an interval of say 10s it gives a nice saw tooth.
Am using ES 2.3.3.
Cheers,
Franek.
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [June 12, 2016, 9:02am UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/2 "2016-06-12T09:02:21Z")
How much heap are you assigning? What JVM? What other settings?
---
### Author:  [@franekrichardson](https://discuss.elastic.co/u/franekrichardson)
#### Post date: [June 12, 2016, 1:49pm UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/3 "2016-06-12T13:49:14Z")
JVM - 1.8.0\_65-b17
Settings are defaults apart from cluster name, the heap I started with default, then extended to 4g and 6g - when watching in visualvm it clearly wasn't recovering all the memory and would eventually hit what ever limit I set.
When I changed the refresh rate to 10s the GC was stable and the 2g heap was never exhausted.
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [June 12, 2016, 9:51pm UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/4 "2016-06-12T21:51:10Z")
How many GB is the 30 million documents?
---
### Author:  [@franekrichardson](https://discuss.elastic.co/u/franekrichardson)
#### Post date: [June 13, 2016, 8:28am UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/5 "2016-06-13T08:28:01Z")
So the latest run inserted ~20 milliion and thats sized at 4.09 GB - pretty small beer 🙂
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [June 13, 2016, 10:39pm UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/6 "2016-06-13T22:39:54Z")
What document structure is it, parent/child, other?
---
### Author:  [@franekrichardson](https://discuss.elastic.co/u/franekrichardson)
#### Post date: [June 14, 2016, 7:38am UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/7 "2016-06-14T07:38:34Z")
juts a flat document - no parent/child or nested documents.
The mapping contains about 6 fields, with some multi fields including unanalyzed data in the index alongside the default analyzed field.
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [June 14, 2016, 7:39am UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/8 "2016-06-14T07:39:45Z")
Can you post mappings and the logs showing OOM?
---
### Author:  [@franekrichardson](https://discuss.elastic.co/u/franekrichardson)
#### Post date: [June 14, 2016, 8:30am UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/9 "2016-06-14T08:30:08Z")
Images from heap usage below - I have the logs and mappings but not sure how to upload them - can you advise please..?
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [June 14, 2016, 8:32am UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/10 "2016-06-14T08:32:21Z")
Use gist/pastebin/etc for logs and mappings.
---
### Author:  [@franekrichardson](https://discuss.elastic.co/u/franekrichardson)
#### Post date: [June 14, 2016, 8:38am UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/11 "2016-06-14T08:38:00Z")
Thanks see below:
[address mapping](http://pastebin.com/j4mipL6f)
[es log](http://pastebin.com/uAKDkquQ)
[es console](http://pastebin.com/pp022Bd0)
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [June 14, 2016, 9:49pm UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/12 "2016-06-14T21:49:24Z")
How big are the bulk requests you are making?
---
### Author:  [@franekrichardson](https://discuss.elastic.co/u/franekrichardson)
#### Post date: [June 15, 2016, 9:53am UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/13 "2016-06-15T09:53:37Z")
1000 records per bulk (note this works fine with a refresh interval of 10s)
---
### Author:  [@franekrichardson](https://discuss.elastic.co/u/franekrichardson)
#### Post date: [June 30, 2016, 1:01pm UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/14 "2016-06-30T13:01:12Z")
Any update on this..?
---
### Author:  [@franekrichardson](https://discuss.elastic.co/u/franekrichardson)
#### Post date: [July 19, 2016, 2:32pm UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/15 "2016-07-19T14:32:11Z")
Hi @warkolm are you planning to look at this..?
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 19, 2016, 9:22pm UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/16 "2016-07-19T21:22:46Z")
I haven't seen anything immediately obvious, sorry.
Maybe someone else can comment as well.
---
### Author:  [@franekrichardson](https://discuss.elastic.co/u/franekrichardson)
#### Post date: [July 25, 2016, 8:52am UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/17 "2016-07-25T08:52:19Z")
ok - should it be raised as a bug then?
---
### Author:  [@andrzejm](https://discuss.elastic.co/u/andrzejm)
#### Post date: [July 27, 2016, 3:24pm UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/18 "2016-07-27T15:24:55Z")
I had OutOfMemory problems with ES 1.7 (trying to index about 10 mln documents).
I had experimented with parameters: a single bulk size (N), sleep time between indexing bulks (S1), and much longer sleep between a few (M) bulks (S2).
I ended with N=5000, S1=1s, M=10, S2=10s.
I'm sure it is very dependent on hardware you have, especially give ES as much memory as you can!
BTW. Observations of usage of CPU, memory and I/O led me to conclusions that ES need some time to process bulks and breaks when usage diagram changes its cyclic nature (for example increased I/O for longer then usual may suggest that ES will break soon - the same on your diagrams).
BTW2. It's a shame that ES breaks with OutOfMemory. It should simply slow down indexing (better strategy IMHO) or throw exception to the caller with the advice "try again later - in about N seconds".
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 10:32pm UTC](https://discuss.elastic.co/t/outofmemory-using-bulk-insert-and-a-refresh-interval-of-1/52458/19 "2017-07-05T22:32:26Z")