Reindex-from-remote error when upgrading from 1.7 -> 5.3


(John Reeder) #1

We are attempting to upgrade our Elasticsearch cluster from V. 1.7.0 to V. 5.3 and when reindexing via the reindex-from-remote procedure are getting the following error:

{
"error": {
"root_cause": [
{
"type": "status_exception",
"reason": "body={"error":"ElasticsearchIllegalArgumentException[Failed to decode scrollId]; nested: IOException[Bad Base64 input character decimal 123 in array position 0]; ","status":400}"
}
],
"type": "status_exception",
"reason": "body={"error":"ElasticsearchIllegalArgumentException[Failed to decode scrollId]; nested: IOException[Bad Base64 input character decimal 123 in array position 0]; ","status":400}",
"caused_by": {
"type": "response_exception",
"reason": "POST http://[source-url]:9200/_search/scroll?scroll=5m: HTTP/1.1 400 Bad Request\n{"error":"ElasticsearchIllegalArgumentException[Failed to decode scrollId]; nested: IOException[Bad Base64 input character decimal 123 in array position 0]; ","status":400}"
}
},
"status": 400
}

The API request that resulted in the above error:

POST _reindex
{
"source": {
"remote": {
"host": "http://[source-url]:9200"
},
"index": "[source-index]"
},
"dest": {
"index": "[destination-index]"
}
}

The same API call made to V. 5.2.2 to the same source server works with no issues. When this is called from 5.3 to a 5.2.2 source previously migrated from the 1.7.0 source it also succeeds. Based on other posts with similar exceptions returned from the source API that reference "scrollId" this appears to possibly be using a feature that was not yet implemented in ES 1.7 but as I mentioned, this Is there a workaround for this or is this a regression that will be fixed in a patch? The production indexes that need to be migrated are quite large and will take quite a bit of time so I really am trying to avoid 1.7.0 -> 5.2.2 -> 5.3.


(Nik Everett) #2

I believe this is a known issue. Entirely my fault:

We need more automated testing for against old versions of Elasticseach for reindex-from-remote.


(John Reeder) #3

Awesome! :smile: Sorry about the duplicate. Has that patch been released? I see it has been merged into your master branch.


(Nik Everett) #4

It is targeted for the 5.3.1 release. We intentionally don't give dates for releases because we really can't be sure but it shouldn't be long now.


(John Reeder) #5

Thanks, I'll keep an eye out for it.


(Rich Horace) #6

Great timing. Looks like released in 5.3.1. Plan on testing next week.


(Rich Horace) #7

Works!! Successfully remote reindex from ES 1.7.1 to ES 5.3.2 :slight_smile:


(system) #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.