I'm the developer of a Grails web application that had embedded Elastic Search 1.x implemented more around 10 years ago (not by me, I've only been the developer for the last 2 yrs).
In the process of upgrading to Grails 5/JDK 11, we looked at bringing ES up to current version as well (1.x -> 2.4 -> 5.6 -> 7.17).
I did originally look at keeping the embedded ES, at least until later this year when I would have more time to look at implementing ES Server (due to embedded no longer being supported). I was able to upgrade to ES 2.4 easily enough, however, going to 5.6 was a lot harder due to the more server-centric approach, and would take a lot of time and effort to achieve.
Instead, we will migrate to ES Server, however, I am struggling to find any documentation on how one would migrate from an embedded ES implementation to a server install.
Would implementing ES 2.4 server and then upgrading from there work or is it more of a case of creating a new ES instance and index from scratch (we have ~6 million records to index)?
Hi @Cyntech,
I haven't used elasticsearch as an embedded app, but I have upgraded an elastic cluster from 1.x up to 7x. Granted that upgrade happened over years as newer versions came out. Instead of doing 3 or more upgrades could you stand up your new ES server (on 7.17) and then use logstash to pull from the old 2.x system and send it on to the 7.17 server? I think you'd have less work to do there, because while 6 million documents is a bit, it doesn't sound too bad. How complex are your documents? How many indices? Of course it all depends on the specifics.
We only have the one index and it's modelled on a core table in our database that has a number of child tables, so while it's not a simple document, it's probably not hugely complex - just a lot of fields.
I have a local replica of our production database, I'll play around with your suggestion on my local machine and see how it goes (I'll try Mark Walkom's suggestion too).
As you goven the step in versions will need to switch to using a client library or different client library I would recommend moving to Elasticsearch 8.6 instead of 7.17 as there has been breaking changes in the Java client and this would make it easier to upgradein the future.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
