AWS elasticsearch service reindexing documents

Elastic Stack Elasticsearch
1

I want to transfer data from one index to another on AWS elasticsearch service. I am using following CURL command. Assume I want to transfer data from index1 to index2. However its not copying all records from index1. Index1 has 26,32,523 documents when I execute following command its copying only 1,41,000 documents.

curl -XPOST 'https://HOST.es.amazonaws.com/_reindex?wait_for_completion=false' -H 'Content-Type: application/json' -d' { "source": { "index": "index1", "size": 10000 }, "dest": { "index": "index2" } }

2

Elastic (the company which hosts and largely staffs these forums) has nothing to do with the AWS Elasticsearch service. AFAIK, they don't support all the APIs that Elasticsearch supports and they seem to have a number of limitations around their service (I think they may cancel reindex commands if you change some cluster settings).

From a format perspective, the curl command you give looks correct. There are some reasons the doc counts may not match, including if some of the docs in index1 don't match the mappings in index2. Logs would indicate this if you could get them.

Anyway, you may want to ask for AWS's support over in the AWS Elasticsearch forums to see if they have any idea what's gone on or you may be interested in using our officially supported Elasticsearch-as-a-service at https://www.elastic.co/cloud.

1 Like
3

Yeah, it was from AWS side, their service it taking time to show updated document count. Thanks for reply.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.