# How to tune ES for maximum write performance

**URL:** <https://discuss.elastic.co/t/how-to-tune-es-for-maximum-write-performance/6601>\
**Category:** Elasticsearch\
**Created:** [February 6, 2012, 8:08am UTC](https://discuss.elastic.co/t/how-to-tune-es-for-maximum-write-performance/6601 "2012-02-06T08:08:25Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Thomas\_Peuss](https://avatars.discourse-cdn.com/v4/letter/t/22d042/32.png) [@Thomas\_Peuss](https://discuss.elastic.co/u/Thomas_Peuss)\
**Post date:** [February 6, 2012, 8:08am UTC](https://discuss.elastic.co/t/how-to-tune-es-for-maximum-write-performance/6601/1 "2012-02-06T08:08:25Z")

</div>

Hello!

Which parameters should be tuned to maximize the write performance of  
ES?

CU  
Thomas

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [February 7, 2012, 10:04am UTC](https://discuss.elastic.co/t/how-to-tune-es-for-maximum-write-performance/6601/2 "2012-02-07T10:04:08Z")

</div>

In general, the built in settings are good to go for high write performance. You might want to increase the index.refresh\_interval to reduce the frequency the index gets refreshed.

On Monday, February 6, 2012 at 10:08 AM, Thomas Peuss wrote:

> Hello!
> 
> Which parameters should be tuned to maximize the write performance of  
> ES?
> 
> CU  
> Thomas

---

<div class="post-metadata">

**Author:** ![Thomas\_Peuss](https://avatars.discourse-cdn.com/v4/letter/t/22d042/32.png) [@Thomas\_Peuss](https://discuss.elastic.co/u/Thomas_Peuss)\
**Post date:** [February 8, 2012, 8:19am UTC](https://discuss.elastic.co/t/how-to-tune-es-for-maximum-write-performance/6601/3 "2012-02-08T08:19:07Z")

</div>

Hello Shay!

On 7 Feb., 11:04, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote:

> In general, the built in settings are good to go for high write performance. You might want to increase the index.refresh\_interval to reduce the frequency the index gets refreshed.

We have set index.refresh\_interval to 120s and it shows good results  
with this setting. Thank you for the hint.

CU  
Thomas

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [February 9, 2012, 7:55am UTC](https://discuss.elastic.co/t/how-to-tune-es-for-maximum-write-performance/6601/4 "2012-02-09T07:55:41Z")

</div>

Note that you can update it in real time using the update settings API.

On Wednesday, February 8, 2012 at 10:19 AM, Thomas Peuss wrote:

> Hello Shay!
> 
> On 7 Feb., 11:04, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> 
> > In general, the built in settings are good to go for high write performance. You might want to increase the index.refresh\_interval to reduce the frequency the index gets refreshed.
> 
> We have set index.refresh\_interval to 120s and it shows good results  
> with this setting. Thank you for the hint.
> 
> CU  
> Thomas

---

<div class="post-metadata">

**Author:** ![Frederic](https://avatars.discourse-cdn.com/v4/letter/f/4491bb/32.png) [@Frederic](https://discuss.elastic.co/u/Frederic)\
**Post date:** [February 9, 2012, 6:10pm UTC](https://discuss.elastic.co/t/how-to-tune-es-for-maximum-write-performance/6601/5 "2012-02-09T18:10:25Z")

</div>

Hi Shay,

We need to have docs available for search asap, while at the same time  
we index 50 docs/sec, 1KB each.

We're using the default refresh\_interval, which is 1sec, right?

Would you recommend decreasing that value (if possible) having such an  
indexing rate? Not sure how much it could affect system load or GC  
times.

We route docs in a 6 nodes, 20 shards, 1 replica cluster. Could the  
addition of more replicas affect indexing times due to the replication  
of info among servers?

Thanks,

On 9 feb, 04:55, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote:

> Note that you can update it in real time using the update settings API.
> 
> On Wednesday, February 8, 2012 at 10:19 AM, Thomas Peuss wrote:
> 
> > Hello Shay!
> 
> > On 7 Feb., 11:04, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> > 
> > > In general, the built in settings are good to go for high write performance. You might want to increase the index.refresh\_interval to reduce the frequency the index gets refreshed.
> 
> > We have set index.refresh\_interval to 120s and it shows good results  
> > with this setting. Thank you for the hint.
> 
> > CU  
> > Thomas

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [February 12, 2012, 11:28am UTC](https://discuss.elastic.co/t/how-to-tune-es-for-maximum-write-performance/6601/6 "2012-02-12T11:28:30Z")

</div>

Yes, the default refresh\_interval is 1 sec. Not sure I understand the question? You can't get to 50 docs per sec indexing rate?

Only the "first" replica (the first additional copy of a shard) will affect the _latency_ of index operation with sync indexing, since it gets replication in parallel to the replicas. Obviously, if you have more replicas, it means more indexing operations happening on the cluster as a whole.

On Thursday, February 9, 2012 at 8:10 PM, Frederic wrote:

> Hi Shay,
> 
> We need to have docs available for search asap, while at the same time  
> we index 50 docs/sec, 1KB each.
> 
> We're using the default refresh\_interval, which is 1sec, right?
> 
> Would you recommend decreasing that value (if possible) having such an  
> indexing rate? Not sure how much it could affect system load or GC  
> times.
> 
> We route docs in a 6 nodes, 20 shards, 1 replica cluster. Could the  
> addition of more replicas affect indexing times due to the replication  
> of info among servers?
> 
> Thanks,
> 
> On 9 feb, 04:55, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> 
> > Note that you can update it in real time using the update settings API.
> > 
> > On Wednesday, February 8, 2012 at 10:19 AM, Thomas Peuss wrote:
> > 
> > > Hello Shay!
> > 
> > > On 7 Feb., 11:04, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> > > 
> > > > In general, the built in settings are good to go for high write performance. You might want to increase the index.refresh\_interval to reduce the frequency the index gets refreshed.
> > 
> > > We have set index.refresh\_interval to 120s and it shows good results  
> > > with this setting. Thank you for the hint.
> > 
> > > CU  
> > > Thomas

---

<div class="post-metadata">

**Author:** ![Frederic](https://avatars.discourse-cdn.com/v4/letter/f/4491bb/32.png) [@Frederic](https://discuss.elastic.co/u/Frederic)\
**Post date:** [February 13, 2012, 2:43pm UTC](https://discuss.elastic.co/t/how-to-tune-es-for-maximum-write-performance/6601/7 "2012-02-13T14:43:29Z")

</div>

Sorry if the question wasn't that clear (as you see, english is not my  
first language 🙂

My point is, if I want docs to searchable, in worst case, sooner than  
in 1 sec and set the refresh\_interval as 0.5, for instance, assuming  
such a value is correct, could that sensibly affect the system load?  
I'm not sure how heavy or resource demanding is the refresh process  
but I guess GC will certainly execute more often.

Thanks for your time and patience,  
Frederic

On 12 feb, 08:28, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote:

> Yes, the default refresh\_interval is 1 sec. Not sure I understand the question? You can't get to 50 docs per sec indexing rate?
> 
> Only the "first" replica (the first additional copy of a shard) will affect the _latency_ of index operation with sync indexing, since it gets replication in parallel to the replicas. Obviously, if you have more replicas, it means more indexing operations happening on the cluster as a whole.
> 
> On Thursday, February 9, 2012 at 8:10 PM, Frederic wrote:
> 
> > Hi Shay,
> 
> > We need to have docs available for search asap, while at the same time  
> > we index 50 docs/sec, 1KB each.
> 
> > We're using the default refresh\_interval, which is 1sec, right?
> 
> > Would you recommend decreasing that value (if possible) having such an  
> > indexing rate? Not sure how much it could affect system load or GC  
> > times.
> 
> > We route docs in a 6 nodes, 20 shards, 1 replica cluster. Could the  
> > addition of more replicas affect indexing times due to the replication  
> > of info among servers?
> 
> > Thanks,
> 
> > On 9 feb, 04:55, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> > 
> > > Note that you can update it in real time using the update settings API.
> 
> > > On Wednesday, February 8, 2012 at 10:19 AM, Thomas Peuss wrote:
> > > 
> > > > Hello Shay!
> 
> > > > On 7 Feb., 11:04, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> > > > 
> > > > > In general, the built in settings are good to go for high write performance. You might want to increase the index.refresh\_interval to reduce the frequency the index gets refreshed.
> 
> > > > We have set index.refresh\_interval to 120s and it shows good results  
> > > > with this setting. Thank you for the hint.
> 
> > > > CU  
> > > > Thomas

---

<div class="post-metadata">

**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\
**Post date:** [February 13, 2012, 6:24pm UTC](https://discuss.elastic.co/t/how-to-tune-es-for-maximum-write-performance/6601/8 "2012-02-13T18:24:17Z")

</div>

Hi Frederic,

We recently worked on improving search performance of an Elasticsearch  
cluster for a client.  
One of the first things we did was increase the refresh\_internal and  
that had a very positive impact on search performance.

## Otis

Search Analytics - [Cloud Monitoring Tools & Services | Sematext](http://sematext.com/search-analytics/index.html)

On Feb 13, 9:43 am, Frederic [focampo...@gmail.com](mailto:focampo...@gmail.com) wrote:

> Sorry if the question wasn't that clear (as you see, english is not my  
> first language 🙂
> 
> My point is, if I want docs to searchable, in worst case, sooner than  
> in 1 sec and set the refresh\_interval as 0.5, for instance, assuming  
> such a value is correct, could that sensibly affect the system load?  
> I'm not sure how heavy or resource demanding is the refresh process  
> but I guess GC will certainly execute more often.
> 
> Thanks for your time and patience,  
> Frederic
> 
> On 12 feb, 08:28, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote:
> 
> > Yes, the default refresh\_interval is 1 sec. Not sure I understand the question? You can't get to 50 docs per sec indexing rate?
> 
> > Only the "first" replica (the first additional copy of a shard) will affect the _latency_ of index operation with sync indexing, since it gets replication in parallel to the replicas. Obviously, if you have more replicas, it means more indexing operations happening on the cluster as a whole.
> 
> > On Thursday, February 9, 2012 at 8:10 PM, Frederic wrote:
> > 
> > > Hi Shay,
> 
> > > We need to have docs available for search asap, while at the same time  
> > > we index 50 docs/sec, 1KB each.
> 
> > > We're using the default refresh\_interval, which is 1sec, right?
> 
> > > Would you recommend decreasing that value (if possible) having such an  
> > > indexing rate? Not sure how much it could affect system load or GC  
> > > times.
> 
> > > We route docs in a 6 nodes, 20 shards, 1 replica cluster. Could the  
> > > addition of more replicas affect indexing times due to the replication  
> > > of info among servers?
> 
> > > Thanks,
> 
> > > On 9 feb, 04:55, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> > > 
> > > > Note that you can update it in real time using the update settings API.
> 
> > > > On Wednesday, February 8, 2012 at 10:19 AM, Thomas Peuss wrote:
> > > > 
> > > > > Hello Shay!
> 
> > > > > On 7 Feb., 11:04, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> > > > > 
> > > > > > In general, the built in settings are good to go for high write performance. You might want to increase the index.refresh\_interval to reduce the frequency the index gets refreshed.
> 
> > > > > We have set index.refresh\_interval to 120s and it shows good results  
> > > > > with this setting. Thank you for the hint.
> 
> > > > > CU  
> > > > > Thomas

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [February 14, 2012, 2:04pm UTC](https://discuss.elastic.co/t/how-to-tune-es-for-maximum-write-performance/6601/9 "2012-02-14T14:04:38Z")

</div>

Setting the refresh interval to a smaller value will cause a higher load on the cluster. If its something your cluster can handle or not is really up to what you define and test.

On Monday, February 13, 2012 at 4:43 PM, Frederic wrote:

> Sorry if the question wasn't that clear (as you see, english is not my  
> first language 🙂
> 
> My point is, if I want docs to searchable, in worst case, sooner than  
> in 1 sec and set the refresh\_interval as 0.5, for instance, assuming  
> such a value is correct, could that sensibly affect the system load?  
> I'm not sure how heavy or resource demanding is the refresh process  
> but I guess GC will certainly execute more often.
> 
> Thanks for your time and patience,  
> Frederic
> 
> On 12 feb, 08:28, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> 
> > Yes, the default refresh\_interval is 1 sec. Not sure I understand the question? You can't get to 50 docs per sec indexing rate?
> > 
> > Only the "first" replica (the first additional copy of a shard) will affect the _latency_ of index operation with sync indexing, since it gets replication in parallel to the replicas. Obviously, if you have more replicas, it means more indexing operations happening on the cluster as a whole.
> > 
> > On Thursday, February 9, 2012 at 8:10 PM, Frederic wrote:
> > 
> > > Hi Shay,
> > 
> > > We need to have docs available for search asap, while at the same time  
> > > we index 50 docs/sec, 1KB each.
> > 
> > > We're using the default refresh\_interval, which is 1sec, right?
> > 
> > > Would you recommend decreasing that value (if possible) having such an  
> > > indexing rate? Not sure how much it could affect system load or GC  
> > > times.
> > 
> > > We route docs in a 6 nodes, 20 shards, 1 replica cluster. Could the  
> > > addition of more replicas affect indexing times due to the replication  
> > > of info among servers?
> > 
> > > Thanks,
> > 
> > > On 9 feb, 04:55, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> > > 
> > > > Note that you can update it in real time using the update settings API.
> > 
> > > > On Wednesday, February 8, 2012 at 10:19 AM, Thomas Peuss wrote:
> > > > 
> > > > > Hello Shay!
> > 
> > > > > On 7 Feb., 11:04, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> > > > > 
> > > > > > In general, the built in settings are good to go for high write performance. You might want to increase the index.refresh\_interval to reduce the frequency the index gets refreshed.
> > 
> > > > > We have set index.refresh\_interval to 120s and it shows good results  
> > > > > with this setting. Thank you for the hint.
> > 
> > > > > CU  
> > > > > Thomas

---

<div class="post-metadata">

**Author:** ![Frederic](https://avatars.discourse-cdn.com/v4/letter/f/4491bb/32.png) [@Frederic](https://discuss.elastic.co/u/Frederic)\
**Post date:** [February 14, 2012, 2:14pm UTC](https://discuss.elastic.co/t/how-to-tune-es-for-maximum-write-performance/6601/10 "2012-02-14T14:14:21Z")

</div>

Fair enough. I just wanted to double check that interval could be reduced  
and what it entails. I'll test it.

Thanks Kimchy and Otis

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:39am UTC](https://discuss.elastic.co/t/how-to-tune-es-for-maximum-write-performance/6601/11 "2017-07-06T03:39:26Z")

</div>