# Bulk Loading Performance slows to a crawl **URL:** \ **Category:** Elasticsearch\ **Created:** [July 14, 2011, 11:40pm UTC](https://discuss.elastic.co/t/bulk-loading-performance-slows-to-a-crawl/4866 "2011-07-14T23:40:28Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![derrickburns](https://avatars.discourse-cdn.com/v4/letter/d/50afbb/32.png) [@derrickburns](https://discuss.elastic.co/u/derrickburns)\ **Post date:** [July 14, 2011, 11:40pm UTC](https://discuss.elastic.co/t/bulk-loading-performance-slows-to-a-crawl/4866/1 "2011-07-14T23:40:28Z")
I am trying to index 80M tweets with a bulk loader that I wrote using the Java API on 0.16.4. When I get to about 2M tweets, indexing performance drops from 4K indexed tweets per second to under 300 tweets per second. CPU load goes from 30% to 3%. Heap memory bounces between 300MB and 600Mb and then spikes to 900Mb. Here are my index settings: ``` index.engine.robin.refresh_interval, 10 indices.memory.index_buffer_size, 0.50 index.number_of_shards, 4 index.number_of_replicas, 0 index.merge.policy.merge_factor, 30 index.merge.policy.use_compound_file, false; index.refresh_interval, "-1" ``` I am doing this on a index I just created on a local node. I am running on a new quad-core i7 Macbook Pro. Ideas??? ---
**Author:** ![derrickburns](https://avatars.discourse-cdn.com/v4/letter/d/50afbb/32.png) [@derrickburns](https://discuss.elastic.co/u/derrickburns)\ **Post date:** [July 14, 2011, 11:51pm UTC](https://discuss.elastic.co/t/bulk-loading-performance-slows-to-a-crawl/4866/2 "2011-07-14T23:51:01Z")
BTW, I noticed over 3000 open files, mostly segment files of the form: /Users/derrickrburns/Documents/workspace/com.rascal.growl/data/ elasticsearch/nodes/0/indices/twitter/0/index/\_0.frq /Users/derrickrburns/Documents/workspace/com.rascal.growl/data/ elasticsearch/nodes/0/indices/twitter/0/index/\_0.prx /Users/derrickrburns/Documents/workspace/com.rascal.growl/data/ elasticsearch/nodes/0/indices/twitter/0/index/\_0.fdt /Users/derrickrburns/Documents/workspace/com.rascal.growl/data/ elasticsearch/nodes/0/indices/twitter/0/index/\_0.fdx /Users/derrickrburns/Documents/workspace/com.rascal.growl/data/ elasticsearch/nodes/0/indices/twitter/0/index/\_0.nrm /Users/derrickrburns/Documents/workspace/com.rascal.growl/data/ elasticsearch/nodes/0/indices/twitter/0/index/\_1.frq On Jul 14, 4:40 pm, Derrick [derrickrbu...@gmail.com](mailto:derrickrbu...@gmail.com) wrote: > I am trying to index 80M tweets with a bulk loader that I wrote using > the Java API on 0.16.4. > > When I get to about 2M tweets, indexing performance drops from 4K > indexed tweets per second to under 300 tweets per second. CPU load > goes from 30% to 3%. Heap memory bounces between 300MB and 600Mb and > then spikes to 900Mb. > > Here are my index settings: > > ``` > index.engine.robin.refresh_interval, 10 > indices.memory.index_buffer_size, 0.50 > index.number_of_shards, 4 > index.number_of_replicas, 0 > index.merge.policy.merge_factor, 30 > index.merge.policy.use_compound_file, false; > index.refresh_interval, "-1" > > ``` > > I am doing this on a index I just created on a local node. > > I am running on a new quad-core i7 Macbook Pro. > > Ideas??? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 15, 2011, 5:17am UTC](https://discuss.elastic.co/t/bulk-loading-performance-slows-to-a-crawl/4866/3 "2011-07-15T05:17:56Z")
- Why are you setting the index.engine.robin.refresh\_interval? Don't set it (you are actually using refresh interval of 10 milliseconds....). The index.refresh\_interval is enough. - How many threads are you indexing with? - I assume its on a single server? Merging will start to kick in and slow things down a bit, thats how it works... - How do you index hte data? Which client? On Friday, July 15, 2011 at 2:51 AM, Derrick wrote: > BTW, I noticed over 3000 open files, mostly segment files of the form: > > /Users/derrickrburns/Documents/workspace/com.rascal.growl/data/ > elasticsearch/nodes/0/indices/twitter/0/index/\_0.frq > /Users/derrickrburns/Documents/workspace/com.rascal.growl/data/ > elasticsearch/nodes/0/indices/twitter/0/index/\_0.prx > /Users/derrickrburns/Documents/workspace/com.rascal.growl/data/ > elasticsearch/nodes/0/indices/twitter/0/index/\_0.fdt > /Users/derrickrburns/Documents/workspace/com.rascal.growl/data/ > elasticsearch/nodes/0/indices/twitter/0/index/\_0.fdx > /Users/derrickrburns/Documents/workspace/com.rascal.growl/data/ > elasticsearch/nodes/0/indices/twitter/0/index/\_0.nrm > /Users/derrickrburns/Documents/workspace/com.rascal.growl/data/ > elasticsearch/nodes/0/indices/twitter/0/index/\_1.frq > > On Jul 14, 4:40 pm, Derrick \<[derrickrbu...@gmail.com](mailto:derrickrbu...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote: > > > I am trying to index 80M tweets with a bulk loader that I wrote using > > the Java API on 0.16.4. > > > > When I get to about 2M tweets, indexing performance drops from 4K > > indexed tweets per second to under 300 tweets per second. CPU load > > goes from 30% to 3%. Heap memory bounces between 300MB and 600Mb and > > then spikes to 900Mb. > > > > Here are my index settings: > > > > index.engine.robin.refresh\_interval, 10 > > indices.memory.index\_buffer\_size, 0.50 > > index.number\_of\_shards, 4 > > index.number\_of\_replicas, 0 > > index.merge.policy.merge\_factor, 30 > > index.merge.policy.use\_compound\_file, false; > > index.refresh\_interval, "-1" > > > > I am doing this on a index I just created on a local node. > > > > I am running on a new quad-core i7 Macbook Pro. > > > > Ideas??? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:00am UTC](https://discuss.elastic.co/t/bulk-loading-performance-slows-to-a-crawl/4866/4 "2017-07-06T04:00:35Z")