I want to migrate a index from es5 cluster to es7 cluster by using remote reindexing.
The number of docs is 5,000,000.
I use curl to call the _reindex api like this:
curl -uelastic:xxx -XPOST -H 'Content-Type: application/json' http://127.0.0.1:9200/_reindex?scroll=60m\&wait_for_completion=false -d'
{
"source": {
"remote": {
"host": "http://1.1.1.1:9200",
"username": "elastic",
"password": "xxx"
},
"index": "my_index",
"size": 8000
},
"dest": {
"index": "my_index"
}
}'
I tried several times but all the tasks failed quickly after 10-20 batches. They have the same error:search_context_missing_exception.
I know the error often occurs when the scroll context is removed for expiring.I have set the scroll=60m and the speed of reindexing is very fast.
"error": { -
"type": "status_exception",
"reason": "body={\"error\":{\"root_cause\":[{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5531730953]\"},{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5531730954]\"},{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5715500036]\"}],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[{\"shard\":-1,\"index\":null,\"reason\":{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5531730953]\"}},{\"shard\":-1,\"index\":null,\"reason\":{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5531730954]\"}},{\"shard\":-1,\"index\":null,\"reason\":{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5715500036]\"}}],\"caused_by\":{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5715500036]\"}},\"status\":404}",
"caused_by": { -
"type": "response_exception",
"reason": "method [POST], host [http://1.1.1.1:9200], URI [/_search/scroll?scroll=3600000000000nanos], status line [HTTP/1.1 404 Not Found]\n{\"error\":{\"root_cause\":[{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5531730953]\"},{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5531730954]\"},{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5715500036]\"}],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[{\"shard\":-1,\"index\":null,\"reason\":{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5531730953]\"}},{\"shard\":-1,\"index\":null,\"reason\":{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5531730954]\"}},{\"shard\":-1,\"index\":null,\"reason\":{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5715500036]\"}}],\"caused_by\":{\"type\":\"search_context_missing_exception\",\"reason\":\"No search context found for id [5715500036]\"}},\"status\":404}"
}