to help see what things we may need to worry about.
Some specific questions
1 - Can the logging.yml file now be done as json like the elasticsearch
one? Sorry, kind of hate the yml format, and json is much easier. here is
hoping....
2 - I know our index mappings need to be updated/checked, but anything pop
out as to really look for? Like, on standard analyzer something was set
before, but now it is gone kind of things?
3 - startup. we have just been running ./bin/elasticsearch. I see that
this has now changed. any recommended ways to have similar function as that
startup?
4 - index formats? Pretty sure that when we upgrade, the indexes are redone
in a new format, yes? Or, if we had static indexes, and we upgrade, are
some in an older format and new ones in the new format?
4a - if they are being re-written, is the limit more IO than anything else?
So, if we had say 1 TB of indexes to do, we should expect the process to
take while to update/write the new index files
5 - any other gotchas people have experienced? Recommend to just go from
.26 -> 1.2.2? Just making sure!
We are looking forward to finally being able to get to this so we can also
check out Marvel on real work being done. Should be fun and thanks for any
insights everyone has!
You probably want to go from 0.20.X to 0.90.12, then to 1.X to be safe.
Ensure you do a optimise (merge) on each index so that they are updated to
the newer lucene version number, then when they are all done move to the
next major version.
to help see what things we may need to worry about.
Some specific questions
1 - Can the logging.yml file now be done as json like the elasticsearch
one? Sorry, kind of hate the yml format, and json is much easier. here is
hoping....
2 - I know our index mappings need to be updated/checked, but anything pop
out as to really look for? Like, on standard analyzer something was set
before, but now it is gone kind of things?
3 - startup. we have just been running ./bin/elasticsearch. I see that
this has now changed. any recommended ways to have similar function as that
startup?
4 - index formats? Pretty sure that when we upgrade, the indexes are
redone in a new format, yes? Or, if we had static indexes, and we upgrade,
are some in an older format and new ones in the new format?
4a - if they are being re-written, is the limit more IO than anything
else? So, if we had say 1 TB of indexes to do, we should expect the process
to take while to update/write the new index files
5 - any other gotchas people have experienced? Recommend to just go from
.26 -> 1.2.2? Just making sure!
We are looking forward to finally being able to get to this so we can also
check out Marvel on real work being done. Should be fun and thanks for any
insights everyone has!
First of all, there is no version 0.26. I am assuming you meant 0.20.6.
Either way, any upgrade from prior of 1.0 to 1.x will require a full
cluster restart.
No clue
Many settings like omit_norms were deprecated, but are still support. I
think that omit_tf has been changed.
I would install via apt-get/yum or use the elasticearch wrapper
Lucene can read indicies up to 1 version behind, so since your index is
based on Lucene 3 and Elasticsearch 1.x is Lucene, you do not need to
upgrade your indices. You should run an optimize on your indices when the
cluster starts back up so that Lucene can upgrade the existing segments.
4a - You can skip the optimize, but you really shouldn't.
One gotcha that I can think about is that all stores are now throttled
with a very low level (IMHO). Increase it based on your hardware and
indexing requirements. The syntax for disabling allocations has change. I'm
sure there are other gotchas, but most settings are still the same.
to help see what things we may need to worry about.
Some specific questions
1 - Can the logging.yml file now be done as json like the elasticsearch
one? Sorry, kind of hate the yml format, and json is much easier. here is
hoping....
2 - I know our index mappings need to be updated/checked, but anything pop
out as to really look for? Like, on standard analyzer something was set
before, but now it is gone kind of things?
3 - startup. we have just been running ./bin/elasticsearch. I see that
this has now changed. any recommended ways to have similar function as that
startup?
4 - index formats? Pretty sure that when we upgrade, the indexes are
redone in a new format, yes? Or, if we had static indexes, and we upgrade,
are some in an older format and new ones in the new format?
4a - if they are being re-written, is the limit more IO than anything
else? So, if we had say 1 TB of indexes to do, we should expect the process
to take while to update/write the new index files
5 - any other gotchas people have experienced? Recommend to just go from
.26 -> 1.2.2? Just making sure!
We are looking forward to finally being able to get to this so we can also
check out Marvel on real work being done. Should be fun and thanks for any
insights everyone has!
Very helpful everyone! (and yes, should have been .20.6!
so, bigger takeaways are
1 - see if we move from 20.6 to .90 and then to 1.2.2 (will add this to our
conversations)
2 - optimize! that is a good one to note. had forgotten the internals of
lucene while talking all of our steps. thanks for reminding me!
Can we run with the same elasticsearch.yml/json file? has anything changed
in the config part that I should be aware of? I know sometimes the
name.value pair naming changes, but, hard to find out what those are
without code digging or trying it and seeing what fails.
On Monday, July 14, 2014 3:37:01 PM UTC-7, Ivan Brusic wrote:
First of all, there is no version 0.26. I am assuming you meant 0.20.6.
Either way, any upgrade from prior of 1.0 to 1.x will require a full
cluster restart.
No clue
Many settings like omit_norms were deprecated, but are still support. I
think that omit_tf has been changed.
I would install via apt-get/yum or use the elasticearch wrapper
Lucene can read indicies up to 1 version behind, so since your index is
based on Lucene 3 and Elasticsearch 1.x is Lucene, you do not need to
upgrade your indices. You should run an optimize on your indices when the
cluster starts back up so that Lucene can upgrade the existing segments.
4a - You can skip the optimize, but you really shouldn't.
One gotcha that I can think about is that all stores are now throttled
with a very low level (IMHO). Increase it based on your hardware and
indexing requirements. The syntax for disabling allocations has change. I'm
sure there are other gotchas, but most settings are still the same.
Cheers,
Ivan
On Mon, Jul 14, 2014 at 3:20 PM, Scott Decker <sc...@publishthis.com
<javascript:>> wrote:
Hey all,
We are going to start our battle plans for upgrading from 0.26 to 1.2.2
and I have been reading through this doc:
to help see what things we may need to worry about.
Some specific questions
1 - Can the logging.yml file now be done as json like the elasticsearch
one? Sorry, kind of hate the yml format, and json is much easier. here is
hoping....
2 - I know our index mappings need to be updated/checked, but anything
pop out as to really look for? Like, on standard analyzer something was set
before, but now it is gone kind of things?
3 - startup. we have just been running ./bin/elasticsearch. I see that
this has now changed. any recommended ways to have similar function as that
startup?
4 - index formats? Pretty sure that when we upgrade, the indexes are
redone in a new format, yes? Or, if we had static indexes, and we upgrade,
are some in an older format and new ones in the new format?
4a - if they are being re-written, is the limit more IO than anything
else? So, if we had say 1 TB of indexes to do, we should expect the process
to take while to update/write the new index files
5 - any other gotchas people have experienced? Recommend to just go from
.26 -> 1.2.2? Just making sure!
We are looking forward to finally being able to get to this so we can
also check out Marvel on real work being done. Should be fun and thanks for
any insights everyone has!
Previously it was unbounded, but now the default is 20mb, which I found to
be extremely low. Also, prior to 1.2, there was a Lucene bug which made
throttling even more limiting:
As far as settings goes, I can't think of any that have changed. Perhaps if
you posted your altered settings, others can give it a once over.
BTW, there is no need for the @ symbol when writing on a mailing list.
Normal English writing rules apply, not the Twitter ones.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
