# Upgrade 0.26.6 -\> 1.2.2 any catches? **URL:** \ **Category:** Elasticsearch\ **Created:** [July 14, 2014, 10:20pm UTC](https://discuss.elastic.co/t/upgrade-0-26-6-1-2-2-any-catches/18665 "2014-07-14T22:20:24Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Scott\_Decker](https://avatars.discourse-cdn.com/v4/letter/s/7bcc69/32.png) [@Scott\_Decker](https://discuss.elastic.co/u/Scott_Decker)\ **Post date:** [July 14, 2014, 10:20pm UTC](https://discuss.elastic.co/t/upgrade-0-26-6-1-2-2-any-catches/18665/1 "2014-07-14T22:20:24Z")
Hey all, We are going to start our battle plans for upgrading from 0.26 to 1.2.2 and I have been reading through this doc: [http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/breaking-changes.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/breaking-changes.html) to help see what things we may need to worry about. Some specific questions 1 - Can the logging.yml file now be done as json like the elasticsearch one? Sorry, kind of hate the yml format, and json is much easier. here is hoping.... 2 - I know our index mappings need to be updated/checked, but anything pop out as to really look for? Like, on standard analyzer something was set before, but now it is gone kind of things? 3 - startup. we have just been running ./bin/elasticsearch. I see that this has now changed. any recommended ways to have similar function as that startup? 4 - index formats? Pretty sure that when we upgrade, the indexes are redone in a new format, yes? Or, if we had static indexes, and we upgrade, are some in an older format and new ones in the new format? 4a - if they are being re-written, is the limit more IO than anything else? So, if we had say 1 TB of indexes to do, we should expect the process to take while to update/write the new index files 5 - any other gotchas people have experienced? Recommend to just go from .26 -\> 1.2.2? Just making sure! We are looking forward to finally being able to get to this so we can also check out Marvel on real work being done. Should be fun and thanks for any insights everyone has! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 14, 2014, 10:31pm UTC](https://discuss.elastic.co/t/upgrade-0-26-6-1-2-2-any-catches/18665/2 "2014-07-14T22:31:42Z")
You probably want to go from 0.20.X to 0.90.12, then to 1.X to be safe. Ensure you do a optimise (merge) on each index so that they are updated to the newer lucene version number, then when they are all done move to the next major version. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 15 July 2014 08:20, Scott Decker [scott@publishthis.com](mailto:scott@publishthis.com) wrote: > Hey all, > We are going to start our battle plans for upgrading from 0.26 to 1.2.2 > and I have been reading through this doc: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/breaking-changes.html) > > to help see what things we may need to worry about. > > Some specific questions > 1 - Can the logging.yml file now be done as json like the elasticsearch > one? Sorry, kind of hate the yml format, and json is much easier. here is > hoping.... > 2 - I know our index mappings need to be updated/checked, but anything pop > out as to really look for? Like, on standard analyzer something was set > before, but now it is gone kind of things? > 3 - startup. we have just been running ./bin/elasticsearch. I see that > this has now changed. any recommended ways to have similar function as that > startup? > 4 - index formats? Pretty sure that when we upgrade, the indexes are > redone in a new format, yes? Or, if we had static indexes, and we upgrade, > are some in an older format and new ones in the new format? > 4a - if they are being re-written, is the limit more IO than anything > else? So, if we had say 1 TB of indexes to do, we should expect the process > to take while to update/write the new index files > 5 - any other gotchas people have experienced? Recommend to just go from > .26 -\> 1.2.2? Just making sure! > > We are looking forward to finally being able to get to this so we can also > check out Marvel on real work being done. Should be fun and thanks for any > insights everyone has! > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624YPvT%3DPu5haH2EhwOzWnEbW%2B6s8kxOEO-zfqMTwiS5UVw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624YPvT%3DPu5haH2EhwOzWnEbW%2B6s8kxOEO-zfqMTwiS5UVw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [July 14, 2014, 10:36pm UTC](https://discuss.elastic.co/t/upgrade-0-26-6-1-2-2-any-catches/18665/3 "2014-07-14T22:36:53Z")
First of all, there is no version 0.26. I am assuming you meant 0.20.6. Either way, any upgrade from prior of 1.0 to 1.x will require a full cluster restart. 1. No clue 2. Many settings like omit\_norms were deprecated, but are still support. I think that omit\_tf has been changed. 3. I would install via apt-get/yum or use the elasticearch wrapper 4. Lucene can read indicies up to 1 version behind, so since your index is based on Lucene 3 and Elasticsearch 1.x is Lucene, you do not need to upgrade your indices. You should run an optimize on your indices when the cluster starts back up so that Lucene can upgrade the existing segments. 4a - You can skip the optimize, but you really shouldn't. 🙂 1. One gotcha that I can think about is that all stores are now throttled with a very low level (IMHO). Increase it based on your hardware and indexing requirements. The syntax for disabling allocations has change. I'm sure there are other gotchas, but most settings are still the same. Cheers, Ivan On Mon, Jul 14, 2014 at 3:20 PM, Scott Decker [scott@publishthis.com](mailto:scott@publishthis.com) wrote: > Hey all, > We are going to start our battle plans for upgrading from 0.26 to 1.2.2 > and I have been reading through this doc: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/breaking-changes.html) > > to help see what things we may need to worry about. > > Some specific questions > 1 - Can the logging.yml file now be done as json like the elasticsearch > one? Sorry, kind of hate the yml format, and json is much easier. here is > hoping.... > 2 - I know our index mappings need to be updated/checked, but anything pop > out as to really look for? Like, on standard analyzer something was set > before, but now it is gone kind of things? > 3 - startup. we have just been running ./bin/elasticsearch. I see that > this has now changed. any recommended ways to have similar function as that > startup? > 4 - index formats? Pretty sure that when we upgrade, the indexes are > redone in a new format, yes? Or, if we had static indexes, and we upgrade, > are some in an older format and new ones in the new format? > 4a - if they are being re-written, is the limit more IO than anything > else? So, if we had say 1 TB of indexes to do, we should expect the process > to take while to update/write the new index files > 5 - any other gotchas people have experienced? Recommend to just go from > .26 -\> 1.2.2? Just making sure! > > We are looking forward to finally being able to get to this so we can also > check out Marvel on real work being done. Should be fun and thanks for any > insights everyone has! > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAoYBqBsx%3DDLe\_cQy66YXY\_UxxrVUNVsJD%2BXr9KMEu7zQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAoYBqBsx%3DDLe_cQy66YXY_UxxrVUNVsJD%2BXr9KMEu7zQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![smonasco\_2](https://avatars.discourse-cdn.com/v4/letter/s/848f3c/32.png) [@smonasco\_2](https://discuss.elastic.co/u/smonasco_2)\ **Post date:** [July 15, 2014, 3:41am UTC](https://discuss.elastic.co/t/upgrade-0-26-6-1-2-2-any-catches/18665/4 "2014-07-15T03:41:26Z")
@Ivan, what do you mean the stores are throttled with a very low level? Do you mean index threads or merge factor or what? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/63273182-325a-4253-bb79-f9a7b5e3b199%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/63273182-325a-4253-bb79-f9a7b5e3b199%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Scott\_Decker](https://avatars.discourse-cdn.com/v4/letter/s/7bcc69/32.png) [@Scott\_Decker](https://discuss.elastic.co/u/Scott_Decker)\ **Post date:** [July 15, 2014, 3:13pm UTC](https://discuss.elastic.co/t/upgrade-0-26-6-1-2-2-any-catches/18665/5 "2014-07-15T15:13:41Z")
Very helpful everyone! (and yes, should have been .20.6! so, bigger takeaways are 1 - see if we move from 20.6 to .90 and then to 1.2.2 (will add this to our conversations) 2 - optimize! that is a good one to note. had forgotten the internals of lucene while talking all of our steps. thanks for reminding me! Can we run with the same elasticsearch.yml/json file? has anything changed in the config part that I should be aware of? I know sometimes the name.value pair naming changes, but, hard to find out what those are without code digging or trying it and seeing what fails. On Monday, July 14, 2014 3:37:01 PM UTC-7, Ivan Brusic wrote: > First of all, there is no version 0.26. I am assuming you meant 0.20.6. > Either way, any upgrade from prior of 1.0 to 1.x will require a full > cluster restart. > > 1. No clue > > 2. Many settings like omit\_norms were deprecated, but are still support. I > think that omit\_tf has been changed. > > 3. I would install via apt-get/yum or use the elasticearch wrapper > > 4. Lucene can read indicies up to 1 version behind, so since your index is > based on Lucene 3 and Elasticsearch 1.x is Lucene, you do not need to > upgrade your indices. You should run an optimize on your indices when the > cluster starts back up so that Lucene can upgrade the existing segments. > > 4a - You can skip the optimize, but you really shouldn't. 🙂 > > 1. One gotcha that I can think about is that all stores are now throttled > with a very low level (IMHO). Increase it based on your hardware and > indexing requirements. The syntax for disabling allocations has change. I'm > sure there are other gotchas, but most settings are still the same. > > Cheers, > > Ivan > > On Mon, Jul 14, 2014 at 3:20 PM, Scott Decker \<[sc...@publishthis.com](mailto:sc...@publishthis.com) > \\> wrote: > > > Hey all, > > We are going to start our battle plans for upgrading from 0.26 to 1.2.2 > > and I have been reading through this doc: > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/breaking-changes.html) > > > > to help see what things we may need to worry about. > > > > Some specific questions > > 1 - Can the logging.yml file now be done as json like the elasticsearch > > one? Sorry, kind of hate the yml format, and json is much easier. here is > > hoping.... > > 2 - I know our index mappings need to be updated/checked, but anything > > pop out as to really look for? Like, on standard analyzer something was set > > before, but now it is gone kind of things? > > 3 - startup. we have just been running ./bin/elasticsearch. I see that > > this has now changed. any recommended ways to have similar function as that > > startup? > > 4 - index formats? Pretty sure that when we upgrade, the indexes are > > redone in a new format, yes? Or, if we had static indexes, and we upgrade, > > are some in an older format and new ones in the new format? > > 4a - if they are being re-written, is the limit more IO than anything > > else? So, if we had say 1 TB of indexes to do, we should expect the process > > to take while to update/write the new index files > > 5 - any other gotchas people have experienced? Recommend to just go from > > .26 -\> 1.2.2? Just making sure! > > > > We are looking forward to finally being able to get to this so we can > > also check out Marvel on real work being done. Should be fun and thanks for > > any insights everyone has! > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/26e360db-953a-48e6-acec-2ecd1eae2ede%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/26e360db-953a-48e6-acec-2ecd1eae2ede%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [July 15, 2014, 3:58pm UTC](https://discuss.elastic.co/t/upgrade-0-26-6-1-2-2-any-catches/18665/6 "2014-07-15T15:58:35Z")
Read more about it here: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Previously it was unbounded, but now the default is 20mb, which I found to be extremely low. Also, prior to 1.2, there was a Lucene bug which made throttling even more limiting: > > > I've been digging into the "merges can fall behind" at high indexing > rates, and â€ĶI discovered some serious issues with the IO throttling, > which we recently (#5902) up'd from 20 MB/sec to 50 MB/sec by default. > > Net/net I think when we ask for 50 MB/sec today we are really > throttling at something like 8 MB/sec! > > Details: > > I indexed a bunch of small log-file type docs into 1 shard, 0 > replicas, using 1 sync \_bulk client, to the point where it did it's > first big-ish merge (611 MB, 440K docs); the merge does not use CFS so > it's really writing 611 MB. I'm using a fast SSD. > > With no throttling (index.store.throttle.type=none), the merge takes > 20.8 seconds. > > With the default 50 MB/sec merge throttling, it takes 72.1 sec, which > far too long (611 MB / 50 = 12.2 sec). The rate limiter enforces the > instantaneous rate, so at worse the merge time should have been 20.8 + > 12.2 = 33 sec but likely much less than that because merging takes > CPU time. > > So I dug in and discovered one problem, I think caused by the > super.flush and then delegate.flush in BufferedChecksumIndexOutput, > where the RateLimiter is always alternately called first on 8192 bytes > then on 0 bytes. If I fix RateLimiter to just ignore those 0 bytes, > the merge time with 50 MB/sec throttle drops to 49.9 sec: better, but > still too long. (I think once we cutover to Lucene's checksums this 0 > byte issue will be fixed?) > > System.nanoTime is actually quite costly, so I suspect the overhead of > just checking whether to pause, and of calling Thread.sleep, is way > too much when the pause time is small. So I change SimpleRateLimiter to > just accumulate the incoming bytes and then once it crosses 1 msec > worth at the specified rate, invoke the pause logic. > > This really improved it: now the merge takes 25.7 sec at 50 MB/sec > throttle, and 64.9 sec at 10 MB/sec throttle. These times seem correct. > > I'll also open a Lucene issue to fix this, and make an XRateLimiter > for ES in the meantime. As far as settings goes, I can't think of any that have changed. Perhaps if you posted your altered settings, others can give it a once over. BTW, there is no need for the @ symbol when writing on a mailing list. Normal English writing rules apply, not the Twitter ones. Cheers, Ivan On Mon, Jul 14, 2014 at 8:41 PM, smonasco [smonasco@gmail.com](mailto:smonasco@gmail.com) wrote: > @Ivan, what do you mean the stores are throttled with a very low level? > Do you mean index threads or merge factor or what? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/63273182-325a-4253-bb79-f9a7b5e3b199%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/63273182-325a-4253-bb79-f9a7b5e3b199%40googlegroups.com) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAc9uaBvsH56y31KnaPHGJ67GMXsuTOsBwF7hn7N7PrCQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAc9uaBvsH56y31KnaPHGJ67GMXsuTOsBwF7hn7N7PrCQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:15am UTC](https://discuss.elastic.co/t/upgrade-0-26-6-1-2-2-any-catches/18665/7 "2017-07-06T01:15:39Z")