Hello,
I'll try to briefly explain the situation:
We have a record which we update once, so we have lots of deleted documents to be left out during merges, however, after a million documents, we perform a rollover, which makes the first index "locked", there will no longer be updates or writes to this index. However, despite a document not being huge, a million documents take up to 70gb, which made me suspicious. After looking up the stats of the index, it turns out about 35gb(total_size_in_bytes) of data is located in "merges", which i assume is because we update every document once.
The index only indicates "19456" deleted documents, which i consider rather weird regarding the merge size. Am I interpreting the statistics wrong or is there something I am doing wrong? Is there a way to reduce the merge size?
Extra info:
My index has 52 segments, would it be intresting to merge it to 1segment or would this be too much data for 1segment? I assume this would be no issue since it will be no longer updated?
Any advice/help is welcome
Kind regards,