We have an index of about 1010^6 documents. The index is dynamic and there are removals as well. In the info we see that there is around 110^6 deleted documents. My understanding was that these documents would be completely removed during bacground merges, but the number seems to be constant for weeks now.
I also understand that es does recommend against manually running optimize to clean up these documents. Any idea what may cause documents not to be deleted?