How to re-index Elasticsearch without stale reads?

Elastic Stack Elasticsearch
1

I have indices with heavy read/write operations. My indices have a read and a write alias. When I need to update the mapping in my indices I go about this process:

  1. create a new index with the new mapping,
  2. add write-alias to the new index.
  3. delete the write-alias to the old index.
  4. reindex the data like this
POST _reindex?wait_for_completion=false
{
"conflicts": "proceed",

"source": {
"index": "old-index"
},
 "dest": {
   "op_type": "create", 
   "index": "new-index"
 }
}
  1. While reindexing read-alias points to old index while the write-alias points to the new index
  2. When the re-indexing is complete, I create a read-alias on the new index and delete the read-alias on the old index.

This process works fine, but there is one caveat. While re-indexing the data is stale to the applications reading, i.e. updates can not be read until I have switched read to the new index.

Since I have quite large indices, the re-indexing takes many hours.

Is there any way to handle re-indexing without reading stale data?

I would of course like to write to two indices at the same time while re-indexing, but as I understand it's not possible.

The only workaround I can think of is to edit on the client-side, so all writes go to both indexes in two separate requests during re-indexing.

Any ideas or comments are much appreciated :pray:

2

Yep that's a fair approach, and the best that I can think of.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.