# Elasticsearch bulk size/performance **URL:** \ **Category:** Elasticsearch\ **Created:** [September 24, 2016, 9:24am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-size-performance/61434 "2016-09-24T09:24:05Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![abhijith\_reddy](https://avatars.discourse-cdn.com/v4/letter/a/977dab/32.png) [@abhijith\_reddy](https://discuss.elastic.co/u/abhijith_reddy)\ **Post date:** [September 24, 2016, 9:24am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-size-performance/61434/1 "2016-09-24T09:24:05Z")
I am trying to load test my elasticsearch instance to figure out the optimal bulk size. Below is my setup - 1 elasticsearch node running the latest (2.4) - 32 GB heap size - 1 index, 1 shard, 0 replicas - refresh interval = -1 - indices.memory.index\_buffer\_size: 30 - index.translog.flush\_threshold\_size:10000 - Mapping is around ~ 20 fields, all not analyzed, stored with lowercase mapping. I tested with 30 parallel workers doing bulk indexing batch sizes of 100, 250, 500, 1000 and each document is roughly around 250 bytes I found that I get the same performance for all the batch sizes, it just takes proportionately longer to index. I get around at around 60k inserts/sec. CPU however increases from ~ 30% to 60% (across all cores). 1. Is this expected ? The documentation suggests to start testing at 5 MB or but when I try that it elasticsearch just takes way too long to respond. 2. what batch should I choose in this case ? I am guessing the lower once as the call returns quickly and consumes less memory. 3. Are there any other settings that I can tweak to get more performance ? 4. I understand performance varies depending on the setup but is 60K/sec reasonable for this setup ? It more than suffices for our use case but I am trying to get a good benchmark. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [September 24, 2016, 10:40am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-size-performance/61434/2 "2016-09-24T10:40:15Z")
> [@abhijith\_reddy](#): > > Is this expected ? The documentation suggests to start testing at 5 MB or but when I try that it elasticsearch just takes way too long to respond. As far as I recall the documentation recommends a maximum bulk size of around 5MB, not to start at that point. A common methodology to determine the optimal bulk size is to start small and increase while throughput keeps improving. In benchmarks I have performed I am usually able to saturate a node with considerably fewer parallel indexing threads, so unless 30 is a requirement, you may want to benchmark with fewer indexing threads as well. > [@abhijith\_reddy](#): > > what batch should I choose in this case ? I am guessing the lower once as the call returns quickly and consumes less memory. Yes, that seems like a good choice. > [@abhijith\_reddy](#): > > Are there any other settings that I can tweak to get more performance ? Elasticsearch 2.x does a lot of optimisations behind the scenes, so there are fewer parameters that needs tuning compared to earlier versions. In the benchmarks I did for [my talk at Elastic{ON}](https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing) I tested with varying number of shards, which can make a difference. > [@abhijith\_reddy](#): > > I understand performance varies depending on the setup but is 60K/sec reasonable for this setup ? It more than suffices for our use case but I am trying to get a good benchmark. That seems to be a good number, but this always depends a lot on the number of CPU, disk performance, type and size of data as well as mappings used. Sustaining max indexing rate does however leave very little resources for querying. I therefore always recommend benchmarking with a combined realistic indexing and query load to find the practical limit for a cluster. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:17pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-size-performance/61434/3 "2017-07-05T22:17:30Z")