Hi,
We are experiencing load and cost issues related to mutable data.
This is our situation:
- Every day we receive 30 million documents we need to index
- 29.9 million of these documents are not queried often and can be stored in warm or colder storage
- These documents get updated multiple times shortly after insertion
- Sporadically, some can be updated days, if not months after the original insertion
- We are only interested in the latest state of each document
We would like to store these 29.9 million mutable documents in a separate index, which can be gradually moved to colder storage to avoid costs.
Any idea on how we could do this?
Kind regards,
Andreas