Adding new documents to production cluster

I have constantly used Elasticsearch cluster with search queries. Once a week i get a batch of new documents i need to add to index. If i add them to index it will greatly slower search speeds while indexing and merging or moving shards.
What is the best way to avoid slowdown?

My solution so far:

  1. Spin up a single node empty elastic.
  2. Restore index i need to update from a snapshot.
  3. Add new documents to this index.
  4. Force merge shards
  5. Snapshot resulting index.
  6. Restore updated index on production cluster.
  7. Update aliases to use updated index and delete old index.

I'm thinking restoring from snapshot shouldn't take much resources. Probably need to warm up restored index for better performance.
Is it normal solution or too complicated?
May be Elasticsearch has proper ways for adding documents without downtime or cluster slowdown?

You could do much the same thing thing, but add the new node to the cluster, make sure you exclude allocation to it for the existing indices, force allocation for the new index to that node, reindex, then move the index off that node and remove it.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.