Update ES2 to ES6 , it is very slowly when we reindex , there have any batter suggest for reindex?

Elastic Stack Elasticsearch
1

update ES2 to ES6 , it is very slowly when we reindex , there have any batter suggest for reindex?
we have two type in ES2 , and have 10T datas; we want update it to ES6 ; it is very slowly when we reindex each type in ES2 to ES6; there have any better way update the ES? please help us.

2

There are no better ways, sorry.

3

ok, there have any config for increase reindex speed ? about 10T , need 20 days for reindex finished. it is slowly than index new data; it is slowly due to rebuild data Structure when update to ES6 ?

4

and there have another problem; when we reindex get error
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Remote responded with a chunk that was too large. Use a smaller batch size."}],"type":"illegal_argument_exception","reason":"Remote responded with a chunk that was too large. Use a smaller batch size.","caused_by":{"type":"content_too_long_exception","reason":"entity content is too long [123362947] for the configured buffer limit [104857600]"}},"status":400}
what is the problem? please help us .

5

If you have still all data present, fill the new index from sratch.

6

There is a couple of things you can do to improve the speed (if you haven't done this already):

  1. Turn off index refreshing, see Bulk Indexing, which lets Elasticsearch focus on writing data rather than reading it back every second to refresh the indices.
  2. Turn off replica shards, see Update Indices Settings, so that Elasticsearch can focus on just writing data to the primary shards. Once the re-indexing is done you can add the replica shards to your indices.

Good luck!

7

thanks, yes, we have config like you say. but is slowly also. another question, should we stop ES index new docs when we do reindex?

8

sorry, what is your mean? can you explain for us.

9

I've had a problem with our index, after upgrading 6.2.4 => 6.3.x.
It was a issue with beats an rigid template.

So instead of reindexing it, I've just created a new index and filled it with data from the begging.
So if you have your data still present than this is also an option.

10

I don't think indexing new documents to the old index will affect the reindex speed very much, unless you index heavily and to the same data nodes the original index is located. In that case the massive IPOS could have an impact. But you could check this yourself by pausing your index operations and just monitor the reindexing speed to see if it increases or not.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.