Improving performance of reindex API?

Elastic Stack Elasticsearch
1

Hi all

Have just started using the reindex API in ES 2.3.5. When it first set off, it was reindexing about 87,000 documents a minute. Over the course of about 10 minutes, this slowed to about 40,000 documents a minute. There is nothing else running on the cluster.

Just wondering if there are any tips/tricks/advice about performance?

Many thanks!

2

Hmm. Now down to 20,000 a minute.

3

With what kind of parameters are you performing the reindex? e.g. Whats the batch size you are reading the "source" index (would be 100 if nothing specified). Also some general idea about your cluster and index sizes might help to give further suggestions.

4

Thanks, Christoph

30 shards spread (at the moment) on 3 servers. ES 2.3.5

Total documents about 500m, 100GB total.

Was using a batch size of 100, am retrying now with a batch size of 10, will report back.

5

Higher batch size is usually better. The default batch size in 5.0 and 2.4 will be 1000 which is fairly reasonable.

Another thing you can do is set the refresh_interval to -1 on the destination index. That usually makes indexing much more efficient. Just do this before the reindex. You can set refresh=true in the reindex request and it'll trigger a refresh when it is done. Don't forget to reset refresh_interval back to something else after you finish with the reindex.

You can also slice the reindex into multiple concurrent requests with something like a range query. Like, issue 10 concurrent reindexes each with non-overlapping range queries.

1 Like
6

Thanks for the tips - that did the trick!

I set the refresh interval to 300s, the batch size to 1000 and I also set replicas to 0. It look like it's going to cut the time down from a few weeks to 8 hours! :slight_smile:

Thanks again!

2 Likes
7

Also a great idea! It is important to remember not to shut off any nodes while that is running or you'll have to start all over which sucks. OTOH _reindex can't survive if you shut down any of the nodes it is ready from anyway so that is probably safe.

1 Like