I have constantly used Elasticsearch cluster with search queries. Once a week i get a batch of new documents i need to add to index. If i add them to index it will greatly slower search speeds while indexing and merging or moving shards.
What is the best way to avoid slowdown?
My solution so far:
- Spin up a single node empty elastic.
- Restore index i need to update from a snapshot.
- Add new documents to this index.
- Force merge shards
- Snapshot resulting index.
- Restore updated index on production cluster.
- Update aliases to use updated index and delete old index.
I'm thinking restoring from snapshot shouldn't take much resources. Probably need to warm up restored index for better performance.
Is it normal solution or too complicated?
May be Elasticsearch has proper ways for adding documents without downtime or cluster slowdown?