We have an elasticsearch index that is having a lot of edits and deletes. This causes the index to have a very high deleteddoc count on all the shards. Based on the recommendation we are assuming ES will take care of it, which it does in most cases. Recently we are running in to situation where a few shard reaches the maximum document count limit of a shard ( 2 Billion plus) and goes in to read only state. In reality, the actual document count is around 1.4 Billion and the Deleted doc cout is 600 Million. The first question is, is the deleted doc count also included in this maximum limit? Our assumption was that it should not be counted.
Second question is, can we use the force merge API with "only_expunge_deletes" option. Does this have any side effect of large segments ( as mentioned in the warning of the forcemerge API).