In Oracle we can do Partition exchange where you can truncate a table and reload data without interrupting user experience. Similarly, I have an index on Elastic Search and want to refresh entire index without impacting users . How can I do this?
To elaborate - I am loading 2 million document via ETL tool to ES. Each time I delete old data and reload today's data. Naturally, deleting and re creating index takes 5 minuted based on my current hardware/network traffic. During this time users are unable to query ES database. How can I solve this problem?
You can do this by introducing an alias. Have this point to the index with the old data while you index the new data into a separate index. When the load has completed you change the alias to point to the index with the new data.
Search an alias instead of an index. Load the new index and then swap the
alias to the new index then delete the index. The alias swap can be done in
1 command.