Index throttling issue

You'd index documents based on their expiry date and store it with the document - so that you could smash the whole index after a while and know "everything in there was expired anyway". And you'd use an alias/alias-ish thing to add a simple date filer I think.

I think if you know when the document will expire up front that is really the way to go for TTL like stuff. But you can build that on the client side.

That doesn't solve the problem where you don't know the expiration up front because changing it would require removing the document from one index and dropping it into another. Which is problematic because refresh times don't line up. I bet someone sufficiently motivated could make TTL more efficient in the single index case by being sneaky with the merge scheduler - never merging segments containing TTLs off by more than an hour, letting segments get very delete-ful without merging if it knows the whole thing will be past its TTL soon. Its fun to think about but it'd be a bunch of work.