Reindex vs Split index speeds

[Pratik_Wadodkar]

Hi Daithi,
Its better to have shard size around 50gb for best performance. If we have index with shard size in TB then it quite difficult to handle when you have take the snapshot or rolling over from one data tier to another data tier.
Split vs reindex:
If you go with reindexing it will take so much time for example like if you want reindex 1GB of data will take around 4-5 minutes so in case of TB data it will gona take days for reindexing on the other hand if you go with split api it will quickly split the index with desired primary shard that you have provide in the split api.
Also one thing when you apply split api make sure you must have good amount of storage because in the begening index try to allocated all the shard on different node and then allocate the data so in this process you might see your storage get incresed by maybe 3-4 times but it will come to its original state by some time.
here are the link for reference

Index stays at massive size after split

Hi, I had mistakenly made a very large index: 1 shard of 406 GB (1.8b documents). Then I split it into 16 shards hoping to get ~25GB sizes. I think the new index triggered a template, since it also got 1 replica shard. The split takes a very long time. I think it's been 5 days and it still wasn't finished. I was running out of space, so I tried disabling replica shards. I checked the _cat/recovery API and it showed no active recoveries. Now it looks like this: index shard prir…

Index size explodes after _split

Recently I had to split an index that had grown to 100GB on a single shard. I used the Split API to split this into 8 shards which nearly instantly created the new index with all 8 shards on the same node. I deleted the old index and created an alias from the new one I noticed the primary store size had grown from 100GB to about 650GB. When enabling replicas and waiting for the cluster to re-balance I ran into out of disk issues over the weekend, causing my entire cluster to go into read-only m…

Cloned/Split Indexes Take Double Disk Space When Increasing Shards

Currently I have some large index's (100GB - 500GB) that have been allocated 5 shards. I would like to increase these to 10 - 20 shards each. If I use the Split API or Clone API it takes 1-2 minutes for the new index to be created, but the disk space doubles if I set it to 10 shards, and quadruples if I set it to 20 shards. (Replicas are still 0). Is there a way to reduce the disk space? The document counts are the same, Im guessing it has to do with how it links to the physical files at th…