Hi,
I am using Elasticsearch 7.8.0. I am trying to copy documents from one index to another using reindex API. I tried 2 cases. Case 1. Only source index is present in the Elasticsearch cluster. Destination index is not pre-created and it got created by reindex API. In this case the documents got successfully copied into destination index.
Case 2. Both source and destination index(pre-created) are present in ES cluster. Now when I run reindex API, even-though the reindex ran without any errors, the documents did not get copied into destination index. After 5-10 minutes all the documents got copied into destination index. I have tried the same command with wait_for_completion=true. But the curl command is coming out and it is not waiting for the process to get completed. I can see that the destination index still has document count 0. After 5-10 minutes the document count got increased.
Note - Source Index size is in kb and it has less documents. But still it is taking 5-10 minutes to copy when I try reindex API as mentioned in Case 2. If I try reindex as mentioned in Case 1 immediately documents got copied into destination index.
But destination index doesn't have any document count. I can see after 5-10 minutes document count got increased.(I have tried Case 2 as mentioned in above post).
Am I missing something here? Please guide me.
[es ~]$ echo $source_index
my-index-1
[es ~]$ echo $dest_index
my-new-index-1
[es ~]$ curl -X POST -k --cert /etc/elasticsearch/certs/clientCrt --key /etc/elasticsearch/certs/clientKey https://elasticsearch:9200/_reindex?wait_for_completion=true -H 'Content-Type: application/json' -d'{ "source": {"index": "'$source_index'" }, "dest": {"index": "'$dest_index'"}}'
{"took":190,"timed_out":false,"total":27,"updated":0,"created":27,"deleted":0,"batches":1,"version_conflicts":0,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1.0,"throttled_until_millis":0,"failures":[]}
[es ~]$ curl -k --cert /etc/elasticsearch/certs/clientCrt --key /etc/elasticsearch/certs/clientKey https://elasticsearch:9200/_cat/indices/my-*?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open my-index-1 KlaxbHBLRZ24PZfsWhp_JQ 1 1 27 0 63.9kb 23.7kb
green open my-new-index-1 kgFum05hQvOD5Rds4--m8g 1 1 0 0 416b 208b
Curl response after 5-10 minutes of reindex API.
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open my-index-1 KlaxbHBLRZ24PZfsWhp_JQ 1 1 27 0 64.1kb 23.8kb
green open my-new-index-1 kgFum05hQvOD5Rds4--m8g 1 1 27 0 47.7kb 23.8kb
Hi,
I can see that the contents of the index got merged as soon as the curl completes. But it is taking time for updating the document count.
Thank you
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.