# Does bulk indexing needs an optimize and the end?

**URL:** <https://discuss.elastic.co/t/does-bulk-indexing-needs-an-optimize-and-the-end/6809>\
**Category:** Elasticsearch\
**Created:** [February 25, 2012, 2:56pm UTC](https://discuss.elastic.co/t/does-bulk-indexing-needs-an-optimize-and-the-end/6809 "2012-02-25T14:56:38Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Onilton\_Maciel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/onilton_maciel/32/2500_2.png) [@Onilton\_Maciel](https://discuss.elastic.co/u/Onilton_Maciel)\
**Post date:** [February 25, 2012, 2:56pm UTC](https://discuss.elastic.co/t/does-bulk-indexing-needs-an-optimize-and-the-end/6809/1 "2012-02-25T14:56:38Z")

</div>

I am bulking indexing something like 50-100 documents every 30s / 1 minute  
(the indexing keeps runnning all the time, because there are always  
documents coming)

Do I need to run optimize after that bulk? Or can I rely on lucene merge  
process?

I know if I am bulk indexing all a whole set of documents like (25  
million), 300 documents for each request, it's good to run optimize at the  
end of all the indexing have finished

But in that situation a mentioned early, do I need to run optimize?

Does bulk indexing segments merge is the same as one document indexing?

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [February 26, 2012, 7:43pm UTC](https://discuss.elastic.co/t/does-bulk-indexing-needs-an-optimize-and-the-end/6809/2 "2012-02-26T19:43:52Z")

</div>

You don't need to run optimize, merge process will happen continuously based on the relevant parameters set.

On Saturday, February 25, 2012 at 4:56 PM, Onilton Maciel wrote:

> I am bulking indexing something like 50-100 documents every 30s / 1 minute (the indexing keeps runnning all the time, because there are always documents coming)
> 
> Do I need to run optimize after that bulk? Or can I rely on lucene merge process?
> 
> I know if I am bulk indexing all a whole set of documents like (25 million), 300 documents for each request, it's good to run optimize at the end of all the indexing have finished
> 
> But in that situation a mentioned early, do I need to run optimize?
> 
> Does bulk indexing segments merge is the same as one document indexing?

---

<div class="post-metadata">

**Author:** ![Onilton\_Maciel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/onilton_maciel/32/2500_2.png) [@Onilton\_Maciel](https://discuss.elastic.co/u/Onilton_Maciel)\
**Post date:** [February 26, 2012, 8:45pm UTC](https://discuss.elastic.co/t/does-bulk-indexing-needs-an-optimize-and-the-end/6809/3 "2012-02-26T20:45:45Z")

</div>

Ok, my doubt was because of this text found on at [elasticsearch.org](http://elasticsearch.org)

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

"Bulk Indexing Usage"

...

Then, once bulk indexing is done, the settings can be updated (back to the

> defaults for example):  
> curl -XPUT localhost:9200/test/\_settings -d '{  
> "index" : {  
> "refresh\_interval" : "1s"  
> }  
> }'  
> And, an optimize should be called:  
> curl -XPOST '[http://localhost:9200/test/\_optimize?max\_num\_segments=5](http://localhost:9200/test/_optimize?max_num_segments=5)'

On Sun, Feb 26, 2012 at 3:43 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:

> You don't need to run optimize, merge process will happen continuously  
> based on the relevant parameters set.
> 
> On Saturday, February 25, 2012 at 4:56 PM, Onilton Maciel wrote:
> 
> I am bulking indexing something like 50-100 documents every 30s / 1 minute  
> (the indexing keeps runnning all the time, because there are always  
> documents coming)
> 
> Do I need to run optimize after that bulk? Or can I rely on lucene merge  
> process?
> 
> I know if I am bulk indexing all a whole set of documents like (25  
> million), 300 documents for each request, it's good to run optimize at the  
> end of all the indexing have finished
> 
> But in that situation a mentioned early, do I need to run optimize?
> 
> Does bulk indexing segments merge is the same as one document indexing?

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [February 26, 2012, 10:08pm UTC](https://discuss.elastic.co/t/does-bulk-indexing-needs-an-optimize-and-the-end/6809/4 "2012-02-26T22:08:08Z")

</div>

Yea, I meant as in bulk indexing a large amount of data, so while you are at it, might make sense to optimize at the end. Not bulk index API 🙂

On Sunday, February 26, 2012 at 10:45 PM, Onilton Maciel wrote:

> Ok, my doubt was because of this text found on at [elasticsearch.org](http://elasticsearch.org) ([http://elasticsearch.org](http://elasticsearch.org))
> 
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-update-settings.html)
> 
> "Bulk Indexing Usage"
> 
> > ...  
> > Then, once bulk indexing is done, the settings can be updated (back to the defaults for example):  
> > curl -XPUT localhost:9200/test/\_settings -d '{  
> > "index" : {  
> > "refresh\_interval" : "1s"  
> > }  
> > }'  
> > And, an optimize should be called:  
> > curl -XPOST '[http://localhost:9200/test/\_optimize?max\_num\_segments=5](http://localhost:9200/test/_optimize?max_num_segments=5)'
> 
> On Sun, Feb 26, 2012 at 3:43 PM, Shay Banon \<[kimchy@gmail.com](mailto:kimchy@gmail.com) ([mailto:kimchy@gmail.com](mailto:kimchy@gmail.com))\> wrote:
> 
> > You don't need to run optimize, merge process will happen continuously based on the relevant parameters set.
> > 
> > On Saturday, February 25, 2012 at 4:56 PM, Onilton Maciel wrote:
> > 
> > > I am bulking indexing something like 50-100 documents every 30s / 1 minute (the indexing keeps runnning all the time, because there are always documents coming)
> > > 
> > > Do I need to run optimize after that bulk? Or can I rely on lucene merge process?
> > > 
> > > I know if I am bulk indexing all a whole set of documents like (25 million), 300 documents for each request, it's good to run optimize at the end of all the indexing have finished
> > > 
> > > But in that situation a mentioned early, do I need to run optimize?
> > > 
> > > Does bulk indexing segments merge is the same as one document indexing?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:38am UTC](https://discuss.elastic.co/t/does-bulk-indexing-needs-an-optimize-and-the-end/6809/5 "2017-07-06T03:38:09Z")

</div>