How to update Lucene version of an huge indices?

Elastic Stack Elasticsearch
1

Hello Geeks,

I have 10 indices of 2 TB size each on ES 7.7 version. Lucene version of those indices is around 7.x. I want to update Lucene version of these indices to latest version that is 8.5?

I am aware of following options:

  • reindexing but it is slow due size.
  • _upgrade api. ES document says it will be removed but I tried running it and it is running. This is also very slow. Took around 4 hours to upgrade one index.

Please suggest me some better way to upgrade lucene version of existing indices.

2

Merging will automatically upgrade the Lucene version AFAIK. You may be able to run a force merge on it to do this, but I would just wait for someone else to confirm this will work :slight_smile:

3

Hi Mark,

All these indices have 100 shards and are already have number of segments to 1.

4

That's a relatively low shard size, have you considered reducing that by half?

I can see you edited your post and added in what you had tried after my response. Given they are 2TB, that sort of time doesn't seem to unreasonable. Why do you need it to run faster?

5

Is it advisable to use _upgrade api which will be removed in future?

I have 10 indices and I am not sure if I can run this api in parallel or not?

I will try to reduce it by half.

6

Any other suggestions?

Another option is reindexing. How can I make reindexing faster? I have 18B docs equivaltent to 2 TB size.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.