Exception while attempting Migration from AWS OSS 1.3 to Elasticsearch 7.17

Elastic Stack Elasticsearch
1

Hello.

We are planning to migrate existing elastic cluster from AWS opensearch service 1.1 to Elastic Cloud: 7.17 with some indexes of ~50GB in size

First, I launched an elastic cloud cluster : 7.17.

As a result, as shown below, the version of OpenSearch is too large and an error occurs if there is no compatibility.

the snapshot was created with Elasticsearch version [35.24.80-beta2] which is higher than the version of this node [7.17.11]
  • Data migration experimented with remote re-indexing from Elastic Cloud:
{
  "source": {
    "remote": {
      "host": https://<aws-clsuter-url>:443,
      "username": "****",
      "password": "*******"
    },
    "index": "<indexName>"
  },
  "dest": {
    "index": "<indexName>"
  }

Result:

"reason" : "body={"error":{"root_cause":"type":"illegal_argument_exception","reason":"No search type for [scan]"}],"type":"illegal_argument_exception","reason":"No search type for [scan]"},"status":400}"

Can someone please suggest if this migration is possible if yes, how can i achieve this.

Thanks in advance

3

Opensearch was based on Elasticsearch 7.10 but has diverged and is no longer compatible. I therefore do not think you can migrate using snapshots, but rather have to create a custom script to reindex your data. I would not expect remote reindexing to be able to connect to Opensearch (unless possibly if you override the version response it provides).

4

Hi @Christian_Dahlqvist ,

Could you please elaborate more on this, as what exactly this custom script would do . Also i have some indexes with ~40GB of data ,would this be a problem of re-indexing ?

5

The script I discussed would basically need to do what a remote reindexing does. If you can get Elasticsearch to connect directly and run reindexing that way it is easier.

Reindexing large data volumes is possible but might be problematic if there is a lack of resources and/or connection/bandwidth issues.

6

Hi @Christian_Dahlqvist ,

Would it be possible for you to share the snippet of script ?

7

I do not have anything to share, so you will need to create it yourself.

8

May be you can use Logstash to do this?

I think that's indeed not possible: Migrate from Amazon OpenSearch Service (formerly Amazon Elasticsearch Service) | Elasticsearch Service Documentation | Elastic

The recent Amazon OpenSearch Service versions do not use an Elasticsearch-compatible repository format, and are also incompatible with the Elasticsearch remote reindex feature.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.