I have an analytics business called Simple Analytics. It's powered by Elasticsearch and it works great. We store analytics data for the websites of our customers. We give our customers to option to delete their account and all their website data.
Currently we store the data in time based indices which works well for performance reasons, but when deleting customer data it's not optimal. Some customers have data that goes a while back. We use deleteByQuery API to delete the specific data of that customer, but it feels like that's not the best way. The deleteByQuery's are quite slow.
I was thinking of creating indices per website combined with time series. This would generate a lot of indices. So that also doesn't feel like the right way.
Who can shine some light on the best way to approach this issue?