# Impact of default indices refresh time on Bulk Indexing

**URL:** <https://discuss.elastic.co/t/impact-of-default-indices-refresh-time-on-bulk-indexing/282441>\
**Category:** Elasticsearch\
**Created:** [August 25, 2021, 5:50am UTC](https://discuss.elastic.co/t/impact-of-default-indices-refresh-time-on-bulk-indexing/282441 "2021-08-25T05:50:05Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pathak18](https://avatars.discourse-cdn.com/v4/letter/p/13edae/32.png) [@Pathak18](https://discuss.elastic.co/u/Pathak18)\
**Post date:** [August 25, 2021, 5:50am UTC](https://discuss.elastic.co/t/impact-of-default-indices-refresh-time-on-bulk-indexing/282441/1 "2021-08-25T05:50:05Z")

</div>

Hi Group,

During Bulk Indexing, when the index.refresh\_interval defaults to 1sec and RefreshPolicy set to NONE.

**Query** : In my case - Save Bulk Request runs in batches with 1k records in each batch. Out of 1k records, suppose 400 records are indexed in 1 sec and Indices are refreshed. Will the first 400 records be available for parallel search query (even though bulk transaction is not complete) ?

**Additional Queries** : What is the optimum setting for index refresh interval? Default is 1sec.  
Is this good to go? Does it become a point of contention later when index size grows ?  
What are the major aspect on deciding the index refresh interval?  
In what all scenarios the default refresh interval of 1 sec can have an adverse impact?  
Does this 1sec Vs 2sec Vs any interval has anything to do with the number of shards or replicas? Please suggest.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 25, 2021, 6:01am UTC](https://discuss.elastic.co/t/impact-of-default-indices-refresh-time-on-bulk-indexing/282441/2 "2021-08-25T06:01:05Z")

</div>

Yep they will.

---

<div class="post-metadata">

**Author:** ![Pathak18](https://avatars.discourse-cdn.com/v4/letter/p/13edae/32.png) [@Pathak18](https://discuss.elastic.co/u/Pathak18)\
**Post date:** [August 25, 2021, 6:07am UTC](https://discuss.elastic.co/t/impact-of-default-indices-refresh-time-on-bulk-indexing/282441/3 "2021-08-25T06:07:40Z")

</div>

Thanks for confirming.

Please help insight on the additional query updated in the section.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 25, 2021, 6:08am UTC](https://discuss.elastic.co/t/impact-of-default-indices-refresh-time-on-bulk-indexing/282441/4 "2021-08-25T06:08:51Z")

</div>

It's not clear what you are asking for there.

---

<div class="post-metadata">

**Author:** ![Pathak18](https://avatars.discourse-cdn.com/v4/letter/p/13edae/32.png) [@Pathak18](https://discuss.elastic.co/u/Pathak18)\
**Post date:** [August 25, 2021, 6:14am UTC](https://discuss.elastic.co/t/impact-of-default-indices-refresh-time-on-bulk-indexing/282441/5 "2021-08-25T06:14:49Z")

</div>

Thank you for quick response. Will reach back if any additional clarification needed.

---

<div class="post-metadata">

**Author:** ![Pathak18](https://avatars.discourse-cdn.com/v4/letter/p/13edae/32.png) [@Pathak18](https://discuss.elastic.co/u/Pathak18)\
**Post date:** [August 25, 2021, 10:06am UTC](https://discuss.elastic.co/t/impact-of-default-indices-refresh-time-on-bulk-indexing/282441/6 "2021-08-25T10:06:37Z")

</div>

What is the optimum setting for index refresh interval? Default is 1sec.  
Does it become a point of contention later when index size grows ?  
What are the major aspect on deciding the index refresh interval?  
In what all scenarios the default refresh interval of 1 sec can have an adverse impact?  
Does this 1sec Vs 2sec Vs any interval has anything to do with the number of shards or replicas? Please suggest.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 25, 2021, 10:07am UTC](https://discuss.elastic.co/t/impact-of-default-indices-refresh-time-on-bulk-indexing/282441/7 "2021-08-25T10:07:24Z")

</div>

What are you trying to solve here?

---

<div class="post-metadata">

**Author:** ![Pathak18](https://avatars.discourse-cdn.com/v4/letter/p/13edae/32.png) [@Pathak18](https://discuss.elastic.co/u/Pathak18)\
**Post date:** [August 25, 2021, 10:24am UTC](https://discuss.elastic.co/t/impact-of-default-indices-refresh-time-on-bulk-indexing/282441/8 "2021-08-25T10:24:00Z")

</div>

Hi Warkolm,

We are building a Search Service solution using search capability of ES. So, to make the service more efficient trying to gauge the pros and cons of default index refresh interval time setting to avoid any performance impact during Bulk Indexing and parallel Search, in clustered environment.

The recent query was to understand the impact and major driving factor on which the default setting to be used **Vs** when it can have adverse affect and needs to be modified to higher no. of seconds. (which is not called out in the documentation).

**Note** : Our indices are going to have documents in the range of 100s of million if not billion.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 25, 2021, 10:05pm UTC](https://discuss.elastic.co/t/impact-of-default-indices-refresh-time-on-bulk-indexing/282441/9 "2021-08-25T22:05:21Z")

</div>

The core answer to all of this is that you need to test based on your expected use case and workloads and find the balance.

However, I don't think that most of what you are worrying about will be that much of an impact.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 22, 2021, 10:06pm UTC](https://discuss.elastic.co/t/impact-of-default-indices-refresh-time-on-bulk-indexing/282441/10 "2021-09-22T22:06:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
