# Index optimization behavior **URL:** \ **Category:** Elasticsearch\ **Created:** [November 7, 2012, 1:40am UTC](https://discuss.elastic.co/t/index-optimization-behavior/9591 "2012-11-07T01:40:12Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Jae](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jae/32/791_2.png) [@Jae](https://discuss.elastic.co/u/Jae)\ **Post date:** [November 7, 2012, 1:40am UTC](https://discuss.elastic.co/t/index-optimization-behavior/9591/1 "2012-11-07T01:40:12Z")
Experimentally, I executed index optimization API but I am not seeing any merging activities in elasticsearch side. I am watching the number of index files in /\/nodex/0/indices//\, I don't see any changes. How can I know what is happening for optimization in elasticsearch? Thank you Best, Jae -- ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [November 7, 2012, 3:48am UTC](https://discuss.elastic.co/t/index-optimization-behavior/9591/2 "2012-11-07T03:48:26Z")
Hi, Hm, I can't think of a direct way ES exposes this. There may be some way, but I can't think of it right now. However, one thing you could look at is disk IO. If optimization is happening, you should see more disk IO if you look at your system monitoring tool or run iostats, vmstat, etc. It's also possible that your index is optimized already, so there is nothing to optimize. ## Otis Search Analytics - [Cloud Monitoring Tools & Services | Sematext](http://sematext.com/search-analytics/index.html) Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) On Tuesday, November 6, 2012 8:40:12 PM UTC-5, Jae wrote: > Experimentally, I executed index optimization API but I am not seeing any > merging activities in elasticsearch side. I am watching the number of index > files in /\/nodex/0/indices//\, I don't see any changes. > > How can I know what is happening for optimization in elasticsearch? > > Thank you > Best, Jae -- ---
**Author:** ![radu\_gheorghe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radu_gheorghe/32/556_2.png) [@radu\_gheorghe](https://discuss.elastic.co/u/radu_gheorghe)\ **Post date:** [November 7, 2012, 9:00am UTC](https://discuss.elastic.co/t/index-optimization-behavior/9591/3 "2012-11-07T09:00:48Z")
Hello Jae, Besides monitoring your IO, you can check out your merges through the Stats API: > **[Elastic — The Search AI Company](https://www.elastic.co)** > > Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight. Try Elastic ... Please note that you would need to explicitly add merge=true as a parameter to get the merge status. Also, you might want to check out the number of segments before and after optimization through the Indices Segments API: > **[Elastic — The Search AI Company](https://www.elastic.co)** > > Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight. Try Elastic ... AFAIK the Optimize API by default only does a merge if it's needed according to your Merge Policy: > **[Elastic — The Search AI Company](https://www.elastic.co)** > > Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight. Try Elastic ... And normally, this is done automatically in background, so in a lot situations just calling the Optimize API wouldn't do anything. You would probably want to specify a rather small number\_of\_segments value there, like 5. But that would be pretty heavy on IO, so you might want to schedule these optimizations for off-peak hours. ## Best regards, Radu [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene On Wed, Nov 7, 2012 at 3:40 AM, Jae [metacret@gmail.com](mailto:metacret@gmail.com) wrote: > Experimentally, I executed index optimization API but I am not seeing any > merging activities in elasticsearch side. I am watching the number of index > files in /\/nodex/0/indices//\, > I don't see any changes. > > How can I know what is happening for optimization in elasticsearch? > > Thank you > Best, Jae > > -- -- ---
**Author:** ![Jae](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jae/32/791_2.png) [@Jae](https://discuss.elastic.co/u/Jae)\ **Post date:** [November 7, 2012, 5:22pm UTC](https://discuss.elastic.co/t/index-optimization-behavior/9591/4 "2012-11-07T17:22:02Z")
Thank you so much for your answers. My bad, I thought the default value of max\_num\_segments was 1 but it wasn't. I didn't specify max\_num\_segments as 1. Do you recommend calling optimize API with max\_num\_segments as 1 or leave it to the default merge function? I am afraid that full optimization can cause something erroneous such as OOM. Best, Jae On Wed, Nov 7, 2012 at 1:00 AM, Radu Gheorghe [radu.gheorghe@sematext.com](mailto:radu.gheorghe@sematext.com) wrote: > Hello Jae, > > Besides monitoring your IO, you can check out your merges through the Stats API: > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-stats.html) > > Please note that you would need to explicitly add merge=true as a > parameter to get the merge status. > > Also, you might want to check out the number of segments before and > after optimization through the Indices Segments API: > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-segments.html) > > AFAIK the Optimize API by default only does a merge if it's needed > according to your Merge Policy: > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/merge.html) > > And normally, this is done automatically in background, so in a lot > situations just calling the Optimize API wouldn't do anything. You > would probably want to specify a rather small number\_of\_segments value > there, like 5. But that would be pretty heavy on IO, so you might want > to schedule these optimizations for off-peak hours. > > ## Best regards, Radu > > [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene > > On Wed, Nov 7, 2012 at 3:40 AM, Jae [metacret@gmail.com](mailto:metacret@gmail.com) wrote: > > > Experimentally, I executed index optimization API but I am not seeing any > > merging activities in elasticsearch side. I am watching the number of index > > files in /\/nodex/0/indices//\, > > I don't see any changes. > > > > How can I know what is happening for optimization in elasticsearch? > > > > Thank you > > Best, Jae > > > > -- > > -- -- ---
**Author:** ![Chris\_Male](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chris_male/32/2607_2.png) [@Chris\_Male](https://discuss.elastic.co/u/Chris_Male)\ **Post date:** [November 8, 2012, 3:49am UTC](https://discuss.elastic.co/t/index-optimization-behavior/9591/5 "2012-11-08T03:49:56Z")
Jae, Optimizing isn't really necessary unless you're worried about file handles or if your data is fairly static. Having multiple segments is not going to cause any problems. I would recommend just letting merging occur. On Thursday, November 8, 2012 4:22:06 AM UTC+11, Jae wrote: > Thank you so much for your answers. > > My bad, I thought the default value of max\_num\_segments was 1 but it > wasn't. I didn't specify max\_num\_segments as 1. > > Do you recommend calling optimize API with max\_num\_segments as 1 or > leave it to the default merge function? > > I am afraid that full optimization can cause something erroneous such as > OOM. > > Best, Jae > > On Wed, Nov 7, 2012 at 1:00 AM, Radu Gheorghe > \<[radu.g...@sematext.com](mailto:radu.g...@sematext.com) \\> wrote: > > > Hello Jae, > > > > Besides monitoring your IO, you can check out your merges through the > > Stats API: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-stats.html) > > > Please note that you would need to explicitly add merge=true as a > > parameter to get the merge status. > > > > Also, you might want to check out the number of segments before and > > after optimization through the Indices Segments API: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-segments.html) > > > AFAIK the Optimize API by default only does a merge if it's needed > > according to your Merge Policy: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/merge.html) > > > > And normally, this is done automatically in background, so in a lot > > situations just calling the Optimize API wouldn't do anything. You > > would probably want to specify a rather small number\_of\_segments value > > there, like 5. But that would be pretty heavy on IO, so you might want > > to schedule these optimizations for off-peak hours. > > > > ## Best regards, Radu > > > > [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene > > > > On Wed, Nov 7, 2012 at 3:40 AM, Jae \<[meta...@gmail.com](mailto:meta...@gmail.com) \\> > > wrote: > > > > > Experimentally, I executed index optimization API but I am not seeing > > > any > > > merging activities in elasticsearch side. I am watching the number of > > > index > > > files in /\/nodex/0/indices//\, > > > I don't see any changes. > > > > > > How can I know what is happening for optimization in elasticsearch? > > > > > > Thank you > > > Best, Jae > > > > > > -- > > > > -- -- ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [November 8, 2012, 10:27am UTC](https://discuss.elastic.co/t/index-optimization-behavior/9591/6 "2012-11-08T10:27:35Z")
hey Jae, optimizing is really a heavy operation and should only be executed if really necessary. if you are doing batch indexing and your index doesn't change frequently then optimizing makes sense. If you do get frequent updates you will be "un-optimized" at the very moment you execute the command. for more details read my blog about optimizing: [http://www.searchworkings.org/blog/-/blogs/simon-says%3A-optimize-is-bad-for-you](http://www.searchworkings.org/blog/-/blogs/simon-says%3A-optimize-is-bad-for-you) simon On Wednesday, November 7, 2012 6:22:06 PM UTC+1, Jae wrote: > Thank you so much for your answers. > > My bad, I thought the default value of max\_num\_segments was 1 but it > wasn't. I didn't specify max\_num\_segments as 1. > > Do you recommend calling optimize API with max\_num\_segments as 1 or > leave it to the default merge function? > > I am afraid that full optimization can cause something erroneous such as > OOM. > > Best, Jae > > On Wed, Nov 7, 2012 at 1:00 AM, Radu Gheorghe > \<[radu.g...@sematext.com](mailto:radu.g...@sematext.com) \\> wrote: > > > Hello Jae, > > > > Besides monitoring your IO, you can check out your merges through the > > Stats API: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-stats.html) > > > Please note that you would need to explicitly add merge=true as a > > parameter to get the merge status. > > > > Also, you might want to check out the number of segments before and > > after optimization through the Indices Segments API: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-segments.html) > > > AFAIK the Optimize API by default only does a merge if it's needed > > according to your Merge Policy: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/merge.html) > > > > And normally, this is done automatically in background, so in a lot > > situations just calling the Optimize API wouldn't do anything. You > > would probably want to specify a rather small number\_of\_segments value > > there, like 5. But that would be pretty heavy on IO, so you might want > > to schedule these optimizations for off-peak hours. > > > > ## Best regards, Radu > > > > [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene > > > > On Wed, Nov 7, 2012 at 3:40 AM, Jae \<[meta...@gmail.com](mailto:meta...@gmail.com) \\> > > wrote: > > > > > Experimentally, I executed index optimization API but I am not seeing > > > any > > > merging activities in elasticsearch side. I am watching the number of > > > index > > > files in /\/nodex/0/indices//\, > > > I don't see any changes. > > > > > > How can I know what is happening for optimization in elasticsearch? > > > > > > Thank you > > > Best, Jae > > > > > > -- > > > > -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:05am UTC](https://discuss.elastic.co/t/index-optimization-behavior/9591/7 "2017-07-06T03:05:26Z")