We are running elastic search on azure with DS4 machines for data nodes (3 data nodes), Memory 28GB, 8TB of premium HDDs (5000 IOPS per disk). We have around 10 indexes and each index has unique type of document. All the indexes together we have 1.4TB of data and recently we had to change index map settings and lead to reindexing. In order to minimize the impact to users we created new indexes and indexing from old to new. We started reindex 5 days back and it still nowhere close to completion. We are not even sure when it is going to complete. We have tried all best practices suggested by community.
We have removed all the replicas for indexes.
We configured refresh interval to -1
We increased the memory for index in heap.
Still there is no improvement at all. We need to complete this as soon as possible and we cannot continue like this without any timelines. Any help on this would be appreciated. Please help with some pointers.