I am currently working on implementing multi-tenancy in Elasticsearch and have come across two prominent approaches: using a shared index across multiple tenants and having a dedicated index per tenant. As part of my research and development efforts, I would like to test a hybrid approach that combines both methods.
Specifically, I am considering creating a custom plugin that includes a scheduler. This scheduler would run every day at midnight to inspect the shared index. The goal is to identify tenants whose data has reached a predefined size. For such tenants, I plan to execute a reindexing operation, moving their data to a new index named "index_tenantId." Subsequently, I aim to remove this data from the shared index.
Given the complexity of the task and the importance of a robust multi-tenancy setup, I would greatly appreciate any suggestions, insights, or advice from the Elasticsearch community. Are there alternative approaches I should consider, or specific best practices that you recommend?
Thank you in advance for your valuable input.
Best regards,