# Lucene 4.0

**URL:** <https://discuss.elastic.co/t/lucene-4-0/9319>\
**Category:** Elasticsearch\
**Created:** [October 11, 2012, 3:18pm UTC](https://discuss.elastic.co/t/lucene-4-0/9319 "2012-10-11T15:18:45Z")\
**Posts on this page:** 7\
**Page:** 2

<div class="post-metadata">

**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\
**Post date:** [March 1, 2013, 5:48pm UTC](https://discuss.elastic.co/t/lucene-4-0/9319/21 "2013-03-01T17:48:57Z")

</div>

Just wondering if any progress has been made on not requiring full cluster  
restarts. Version 1.0? 🙂

--  
Ivan

On Thu, Oct 11, 2012 at 6:16 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:

> Heya, few points:
> 
> Indexes in elasticsearch has been "backward" compatible, and when  
> upgrading to Lucene 4.0, it will be backward compatible as well (both on  
> the Lucene level, and on the ES level). You might need to do an "upgrade"  
> to make use of newer versions, or to make sure there is no "emulation"  
> layer on Lucene level.
> 
> So, upgrading to 0.20 and 0.21 will be compatible index wise (no need to  
> reindex), but you will need to do a full cluster restart (though we are  
> working on eventually not needing that as well, first infrastructure for  
> that is already going to be in upcoming 0.20).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [March 5, 2013, 12:46am UTC](https://discuss.elastic.co/t/lucene-4-0/9319/22 "2013-03-05T00:46:24Z")

</div>

Between major versions, we hope to nail it for 1.0, but the plan is not to hold 1.0 is we won't have done by that time. We'll see...

On Mar 1, 2013, at 9:48 AM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote:

> Just wondering if any progress has been made on not requiring full cluster restarts. Version 1.0? 🙂
> 
> --  
> Ivan
> 
> On Thu, Oct 11, 2012 at 6:16 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:  
> Heya, few points:
> 
> Indexes in elasticsearch has been "backward" compatible, and when upgrading to Lucene 4.0, it will be backward compatible as well (both on the Lucene level, and on the ES level). You might need to do an "upgrade" to make use of newer versions, or to make sure there is no "emulation" layer on Lucene level.
> 
> So, upgrading to 0.20 and 0.21 will be compatible index wise (no need to reindex), but you will need to do a full cluster restart (though we are working on eventually not needing that as well, first infrastructure for that is already going to be in upcoming 0.20).
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\
**Post date:** [March 5, 2013, 1:12am UTC](https://discuss.elastic.co/t/lucene-4-0/9319/23 "2013-03-05T01:12:41Z")

</div>

Thanks for following up on this thread. The obvious next question is will  
1.0 be the successor to 0.90?

--  
Ivan

On Mon, Mar 4, 2013 at 4:46 PM, [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:

> Between major versions, we hope to nail it for 1.0, but the plan is not to  
> hold 1.0 is we won't have done by that time. We'll see...
> 
> On Mar 1, 2013, at 9:48 AM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote:
> 
> Just wondering if any progress has been made on not requiring full cluster  
> restarts. Version 1.0? 🙂
> 
> --  
> Ivan
> 
> On Thu, Oct 11, 2012 at 6:16 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> 
> > Heya, few points:
> > 
> > Indexes in elasticsearch has been "backward" compatible, and when  
> > upgrading to Lucene 4.0, it will be backward compatible as well (both on  
> > the Lucene level, and on the ES level). You might need to do an "upgrade"  
> > to make use of newer versions, or to make sure there is no "emulation"  
> > layer on Lucene level.
> > 
> > So, upgrading to 0.20 and 0.21 will be compatible index wise (no need to  
> > reindex), but you will need to do a full cluster restart (though we are  
> > working on eventually not needing that as well, first infrastructure for  
> > that is already going to be in upcoming 0.20).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [March 5, 2013, 4:42am UTC](https://discuss.elastic.co/t/lucene-4-0/9319/24 "2013-03-05T04:42:57Z")

</div>

Yes, 1.0 will be the next major release post 0.90. Will post our thoughts on which features we wish to try and get into 1.0 once we release 0.90.

On Mar 4, 2013, at 5:12 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote:

> Thanks for following up on this thread. The obvious next question is will 1.0 be the successor to 0.90?
> 
> --  
> Ivan
> 
> On Mon, Mar 4, 2013 at 4:46 PM, [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:  
> Between major versions, we hope to nail it for 1.0, but the plan is not to hold 1.0 is we won't have done by that time. We'll see...
> 
> On Mar 1, 2013, at 9:48 AM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote:
> 
> > Just wondering if any progress has been made on not requiring full cluster restarts. Version 1.0? 🙂
> > 
> > --  
> > Ivan
> > 
> > On Thu, Oct 11, 2012 at 6:16 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:  
> > Heya, few points:
> > 
> > Indexes in elasticsearch has been "backward" compatible, and when upgrading to Lucene 4.0, it will be backward compatible as well (both on the Lucene level, and on the ES level). You might need to do an "upgrade" to make use of newer versions, or to make sure there is no "emulation" layer on Lucene level.
> > 
> > So, upgrading to 0.20 and 0.21 will be compatible index wise (no need to reindex), but you will need to do a full cluster restart (though we are working on eventually not needing that as well, first infrastructure for that is already going to be in upcoming 0.20).
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Mohammady\_Mahdy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohammady_mahdy/32/2128_2.png) [@Mohammady\_Mahdy](https://discuss.elastic.co/u/Mohammady_Mahdy)\
**Post date:** [March 5, 2013, 10:14am UTC](https://discuss.elastic.co/t/lucene-4-0/9319/25 "2013-03-05T10:14:18Z")

</div>

Can you add an option that disables this at the cost of making your indices  
not backwards compatible? (but faster/ memory friendlier)  
Is there a way of completely upgrading your index planned?

On Friday, October 12, 2012 12:26:33 PM UTC+4, simonw wrote:

> On Thursday, October 11, 2012 10:40:53 PM UTC+2, Ivan Brusic wrote:
> 
> > Lucene is always backwards compatible within one version, so Lucene 4.0  
> > code should be able to read a 3.x index. Writing indexes with a newer  
> > version and reading them in an older one is more problematic. That said, a  
> > new ES version is more than just a Lucene upgrade, so it might require a  
> > reindex.
> 
> this question has been asked a couple of times so let me elaborate on this  
> a little for those who are interested what the main differences are between  
> 4.0 and 3.x on the index level. It is correct in general that lucene 4.0  
> can read 3.0 indexes but this comes with a cost this time. in lucene 4.0 we  
> changed the sort order from UTF-16 to UTF-8 on the lowest level so 3.x  
> indices are sorted "differently". To make this still work with the 4.0 API  
> we added a "re-mapping" layer for surrogate characters to maintain the  
> correct sort order. This will have some cost in performance but it should  
> not be dramatic. Yet, it is still a good idea to upgrade the index. The  
> good news is lucene can by-itself do that in the background. If you merge a  
> 3.x segment with 4.0 it will be merged into a 4.0 segment so you can  
> "upgrade-over-time". Anyhow, in practice ES users should be need this  
> knowledge and we will provide flexible ways to upgrade to an ES version  
> that runs lucene 4.0.
> 
> simon
> 
> > ES version incompatibility is always due to the internal communication  
> > API changing.
> > 
> > --  
> > Ivan
> > 
> > On Thu, Oct 11, 2012 at 11:20 AM, Eugene Strokin [eug...@strokin.info](mailto:eug...@strokin.info)wrote:
> > 
> > > Shay, thanks for update.  
> > > Just wander about back compatibility.  
> > > If switching from 0.19 to 0.20 will it require reindexing, or we could  
> > > just update the libs?  
> > > And from 20 to 21, I guess it will require reindexing, but want to hear  
> > > confirmation from you.
> > > 
> > > Thank you
> > > 
> > > On Thursday, October 11, 2012 11:18:46 AM UTC-4, kimchy wrote:
> > > 
> > > > Heya fellows,
> > > > 
> > > > ```
> > > > Lucene 4.0 is out the door, wanted to update you all about how we 
> > > > 
> > > > ```
> > > > 
> > > > plan to upgrade to it on elasticsearch. The plan is to first release 0.20  
> > > > version still using Lucene 3.6.x, this should happen in the next week or  
> > > > so. The reason is that we want to get 0.20 features at the hand of users as  
> > > > fast as possible, without waiting for the upgrade to 4.0.
> > > > 
> > > > ```
> > > > We will also start the process of upgrading the 4.0, which will be 
> > > > 
> > > > ```
> > > > 
> > > > in the next major elasticsearch version. This will include upgrading to  
> > > > 4.0, and making use of the new features and exposing them to the users.  
> > > > This shouldn't take too long, though we do want to see 4.0.0 GA Lucene "out  
> > > > in the wild" a bit before releasing a formal release (0.21) of  
> > > > elasticsearch with it.
> > > > 
> > > > -shay.banon
> > > 
> > > --

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\
**Post date:** [March 6, 2013, 11:50am UTC](https://discuss.elastic.co/t/lucene-4-0/9319/26 "2013-03-06T11:50:14Z")

</div>

On Tuesday, March 5, 2013 11:14:18 AM UTC+1, Mo wrote:

> Can you add an option that disables this at the cost of making your  
> indices not backwards compatible? (but faster/ memory friendlier)  
> Is there a way of completely upgrading your index planned?

I am not sure what you mean by "disables this"? If you want to upgrade your  
entire index to Lucene 4.0 you need to run an optimze on the index which  
will re-write the index in the new format.

simon

> On Friday, October 12, 2012 12:26:33 PM UTC+4, simonw wrote:
> 
> > On Thursday, October 11, 2012 10:40:53 PM UTC+2, Ivan Brusic wrote:
> > 
> > > Lucene is always backwards compatible within one version, so Lucene 4.0  
> > > code should be able to read a 3.x index. Writing indexes with a newer  
> > > version and reading them in an older one is more problematic. That said, a  
> > > new ES version is more than just a Lucene upgrade, so it might require a  
> > > reindex.
> > 
> > this question has been asked a couple of times so let me elaborate on  
> > this a little for those who are interested what the main differences are  
> > between 4.0 and 3.x on the index level. It is correct in general that  
> > lucene 4.0 can read 3.0 indexes but this comes with a cost this time. in  
> > lucene 4.0 we changed the sort order from UTF-16 to UTF-8 on the lowest  
> > level so 3.x indices are sorted "differently". To make this still work with  
> > the 4.0 API we added a "re-mapping" layer for surrogate characters to  
> > maintain the correct sort order. This will have some cost in performance  
> > but it should not be dramatic. Yet, it is still a good idea to upgrade the  
> > index. The good news is lucene can by-itself do that in the background. If  
> > you merge a 3.x segment with 4.0 it will be merged into a 4.0 segment so  
> > you can "upgrade-over-time". Anyhow, in practice ES users should be need  
> > this knowledge and we will provide flexible ways to upgrade to an ES  
> > version that runs lucene 4.0.
> > 
> > simon
> > 
> > > ES version incompatibility is always due to the internal communication  
> > > API changing.
> > > 
> > > --  
> > > Ivan
> > > 
> > > On Thu, Oct 11, 2012 at 11:20 AM, Eugene Strokin [eug...@strokin.info](mailto:eug...@strokin.info)wrote:
> > > 
> > > > Shay, thanks for update.  
> > > > Just wander about back compatibility.  
> > > > If switching from 0.19 to 0.20 will it require reindexing, or we could  
> > > > just update the libs?  
> > > > And from 20 to 21, I guess it will require reindexing, but want to hear  
> > > > confirmation from you.
> > > > 
> > > > Thank you
> > > > 
> > > > On Thursday, October 11, 2012 11:18:46 AM UTC-4, kimchy wrote:
> > > > 
> > > > > Heya fellows,
> > > > > 
> > > > > ```
> > > > > Lucene 4.0 is out the door, wanted to update you all about how we 
> > > > > 
> > > > > ```
> > > > > 
> > > > > plan to upgrade to it on elasticsearch. The plan is to first release 0.20  
> > > > > version still using Lucene 3.6.x, this should happen in the next week or  
> > > > > so. The reason is that we want to get 0.20 features at the hand of users as  
> > > > > fast as possible, without waiting for the upgrade to 4.0.
> > > > > 
> > > > > ```
> > > > > We will also start the process of upgrading the 4.0, which will be 
> > > > > 
> > > > > ```
> > > > > 
> > > > > in the next major elasticsearch version. This will include upgrading to  
> > > > > 4.0, and making use of the new features and exposing them to the users.  
> > > > > This shouldn't take too long, though we do want to see 4.0.0 GA Lucene "out  
> > > > > in the wild" a bit before releasing a formal release (0.21) of  
> > > > > elasticsearch with it.
> > > > > 
> > > > > -shay.banon
> > > > 
> > > > --

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:48am UTC](https://discuss.elastic.co/t/lucene-4-0/9319/27 "2017-07-06T02:48:06Z")

</div>

Closed on July 6, 2017, 2:48am UTC



[Previous page](https://discuss.elastic.co/t/lucene-4-0/9319.md?page=1)
