How to migrate old elastic search index to new version of elastic search?


(Sumit Guptaa) #1

hi all,

Essentially I cannot find documents or resources that explains the procedure of upgrading a running Elastic search instance into the current version.

Please help me out in a few scenarios:-

1 If I am running an Elastic search instance in a single server, how do I upgrade the instance and not lose data?

2 If I am running multiple Elasticsearch instances in a number of servers, how do I keep my operations running, while I upgrade my Elasticsearch instances without losing data?

if there are proper procedures or explanations on this it will greatly help my understanding and work. Thanks!

Regards,
Sumit Gupta


(sujoysett) #2

Hi,

I have recently upgraded my ES indices from elasticsearch 0.18.6 to 0.19.2,
and the procedure followed was:

  • Stop indexing operations on the indices, run refresh and flush on the
    indices (at cluster level)
  • Stop all the nodes.
  • Upgrade all the nodes to next version. (In my case data location was
    separate, so no copying of data needed, only in the new version config file
    I had to mention the existing data location.)
  • Start the nodes (new version).

The above steps will work for case 1. But there will be a downtime.

For case 2, a more complex procedure is available, but I have NOT tried it
personally at production level. I had success with it at Development stage.

  • First ensure replicas of all the shards exists in all the nodes,
    change replica count if necessary for this and wait until green.
  • Of the multiple available nodes, shut down some, and change the
    cluster name in configuration file as well as in local data folder.
  • This will prevent these nodes from interfering with existing active
    ones due to mismatch in cluster name. Proceed with upgrading these set of
    nodes while the others serve your application.
  • Once you make these upgraded nodes running, shut down the rest and
    proceed for upgrading them also.

These steps worked for us as we were using pure http client (no java API,
so changing cluster name did not create problem), and using
commons-httpclient-failover for the requests can help in switching over
automatically. However, most of our operations were search based, and it
created no problem, there might be data mismatch if indexing operations
remain active during this activity, I am not sure.

Please note that the second solution posted above is only a trial workout,
worked in my case, and not a general solution.

The link at location
http://stackoverflow.com/questions/9497223/how-to-upgrade-a-running-elasticsearch-older-instance-to-a-newer-version was
helpful.

On Monday, May 28, 2012 10:42:38 AM UTC+5:30, Sumit Gupta wrote:

hi all,

Essentially I cannot find documents or resources that explains the
procedure
of upgrading a running Elastic search instance into the current version.

Please help me out in a few scenarios:-

1 If I am running an Elastic search instance in a single server, how do I
upgrade the instance and not lose data?

2 If I am running multiple Elasticsearch instances in a number of servers,
how do I keep my operations running, while I upgrade my Elasticsearch
instances without losing data?

if there are proper procedures or explanations on this it will greatly
help
my understanding and work. Thanks!

Regards,
Sumit Gupta

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/how-to-migrate-old-elastic-search-index-to-new-version-of-elastic-search-tp4018412.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(Sumit Guptaa) #3

hi sujoysett

can you tell me what is the means of stop all nodes and how we can stop all nodes,and i want to ask one more things if we migrate our data from Elastic Search 0.18.7 to elastic search 0.19.0 then the data that is store in elastic search 0.18.7 is store or not?

thanx

Regards,
Sumit Gupta


(pulkitsinghal) #4

I'll try to bridge the gap: nodes means servers

On Monday, May 28, 2012 4:02:08 AM UTC-5, Sumit Gupta wrote:

hi sujoysett

can you tell me what is the means of stop all nodes and how we can stop
all
nodes,and i want to ask one more things if we migrate our data from
Elastic
Search 0.18.7 to elastic search 0.19.0 then the data that is store in
elastic search 0.18.7 is store or not?

thanx

Regards,
Sumit Gupta

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/how-to-migrate-old-elastic-search-index-to-new-version-of-elastic-search-tp4018412p4018426.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(sujoysett) #5

Yes, by nodes I meant running instances of elasticsearch.
And for the second question, yes, that is what the term migration means.
Keeping your existing data as it is, but using the same on a new
elasticsearch version.

Sujoy.

On Monday, May 28, 2012 6:23:56 PM UTC+5:30, pulkitsinghal wrote:

I'll try to bridge the gap: nodes means servers

On Monday, May 28, 2012 4:02:08 AM UTC-5, Sumit Gupta wrote:

hi sujoysett

can you tell me what is the means of stop all nodes and how we can stop
all
nodes,and i want to ask one more things if we migrate our data from
Elastic
Search 0.18.7 to elastic search 0.19.0 then the data that is store in
elastic search 0.18.7 is store or not?

thanx

Regards,
Sumit Gupta

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/how-to-migrate-old-elastic-search-index-to-new-version-of-elastic-search-tp4018412p4018426.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(Sumit Guptaa) #6

thanx sujoysett


(system) #7