How to create a 1.7 Snapshot compatible with 1.5.2

Elastic Stack Elasticsearch
1

We've been running Elasticsearch ourselves (1.7.0) for a while. We want now to use AWS ES which is at version 1.5.2

When generating a snapshot and restoring from it into AWS ES, we receive the following error from AWS ES:

{"error":"SnapshotRestoreException[[perfsnapshot:snapshot] the snapshot was created with Elasticsearch version [1.7.0] which is higher than the version of this node [1.5.2]]","status":500

(which is obviously an elasticsearch error)

Hence my question is, how do I go about transferring data from a 1.7.0 elasticsearch to a 1.5.2 elasticsearch?

Any help gratefully appreciated

2

You have to reindex.

3

Thanks for the quick reply, I read as much on other threads, but I'm not sure all the steps involved in this. Does this mean I have to export the raw data somehow, and then import into a 1.5.2 elasticsearch and then "reindex" on the 1.5.2 Elasticsearch, and then create a snapshot, and finally restore into AWS ES? Or is it much more simple than that?

Otherwise I dont really quite understand what to do from your answer.. sorry!

4

You have to read the data again from the source or from elasticsearch (_source) and then push the JSON again to another cluster (reindex that is).

You can use logstash for example. http://david.pilato.fr/blog/2015/05/20/reindex-elasticsearch-with-logstash/

5

Many thanks..

In the end I found this tool: https://github.com/garbin/elasticsearch-reindex - which looks like it makes it easy to reindex from my source ES cluster (1.7.0) to my destination ES cluster (1.5.2) . For other people's reference, after installing nodejs, npm and installing this tool, I did this:

export INDEXES=$( curl --silent http://${SOURCE_ES}:9200/_cat/indices | awk '{print $3}' )
export TYPES=$( for i in $INDEXES ; do curl --silent http://${SOURCE_ES}:9200/$i | jq -r '. | .["'$i'"] | .mappings | keys[]' | sed -e "s/^/$i://" )

for TYPE in $TYPES ; do elasticsearch-reindex -f http://${SOURCE_ES}:9200/$TYPE -t http://${DESTINATION_ES}:9200 ; done

It seems to be working so far, it'll take a day or so, so I'll report back if it didn't work - thanks to the links in your blog article David and for the pointer in the right direction. Obviously this only reindexes indexes, and doesn't create e.g. index aliases, but its definitely a start

6

In case you are looking for an elasticsearch as a service solution (and not specifically aws), you can look at found which supports 1.7 and 2.x series.