We have a shell script which takes the name of an index and then reindexes it.
We are using ES 7.17.0
The reindex command-
response=$(curl -u $CREDENTIALS -X POST "$PROTOCOL://$HOST:9200/_reindex?slices=50&refresh&wait_for_completion=false" -H "Content-Type: application/json" -d "{ \"source\": { \"index\": \"$INDEX\",\"size\":5}, \"dest\": { \"index\": \"$new_index\" } }" --write-out '%{http_code}')
Here we are using slice and size as well.But i was getting the search context missing exception and es was complaining that no search context is found for most of the slices.
The index is big and has a lot of docs.I figured out that search context expires in 5 min.Hence i am giving scroll=1h in the reindex call
But there are many indices for which it might take even more time.So instead of hardcoding it everytime i was thinking of giving it as scroll=2d
I guess this might slow down the performance of Elasticsearch as if the reindexing is completed soon,still search context will be active..
I am thinking of using the following approach-
Make scroll=2d and then if the reindexing is complete anytime sooner than clear the search context using this api-
DELETE /_search/scroll
{
"scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ=="
}
Will this approach work and is this the correct approach?
I am a bit skeptical because till now we have not been touching the search context explicitly and i want to make sure clearing this doesn't lead to any other error..
Also should i use size as auto instead of a manual number?
Any other approach to solve this issue is welcome.