# Reindexing Strategy **URL:** \ **Category:** Elasticsearch\ **Created:** [August 29, 2012, 5:54pm UTC](https://discuss.elastic.co/t/reindexing-strategy/8884 "2012-08-29T17:54:46Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![stratawing](https://avatars.discourse-cdn.com/v4/letter/s/9de0a6/32.png) [@stratawing](https://discuss.elastic.co/u/stratawing)\ **Post date:** [August 29, 2012, 5:54pm UTC](https://discuss.elastic.co/t/reindexing-strategy/8884/1 "2012-08-29T17:54:46Z")
I have a need to update my mappings and will therefore need to reindex - but will be doing so directly from an existing ES index (I'm using ES as my data store). I've read other posts, and think the following strategy will work, but have a couple questions: Here's the strategy: 1 - create new index with new mappings and different index name 2 - extract data from old index (e.g., using scroll) 3 - bulk load from old index into new index 4 - confirm consistency between indices (how do I do this?) 5 - add 'alias' to new index to map old index name to new index 6 - delete old index. I have two questions regarding the strategy above: 1 - I'm not clear on how to confirm that the new index is fully consistent with the old index. Any suggestions? Understood that I may need to halt writes to the old index while the comparison is being performed. 2 - There will be a moment (however brief) where both the new index (with the alias) and old index are getting requests. Once the alias is in place, will search results against both indices create duplicates in the result set (i.e., the same doc will show up twice)? If so - it might create problems in my application. Probably a minimal concern - but I'd like to hear whether anyone else has had issues "swapping" indices using the alias functionality in this manner. Many thanks in advance for your input. -- ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [August 29, 2012, 10:30pm UTC](https://discuss.elastic.co/t/reindexing-strategy/8884/2 "2012-08-29T22:30:20Z")
For question 1, I am unsure about what your definition of "consistency" might be. The availability of documents? The number of documents/fields? The relevancy of certain searches? Changing your mapping will affect your index and by definition, it will not be the same as the original one. Consistency in the data world usual means the consistent propagation of changes. Updating a mapping would not affect that. Question 2: I use aliases for searching/incremental-updates, real index names for full indexing. Once a new index is created, you can create an atomic transaction that will remove the alias from the old index and create an alias on the new one. Searches only know about the alias, never the real index name. Cheers, Ivan On Wed, Aug 29, 2012 at 10:54 AM, stratawing [stratawing@gmail.com](mailto:stratawing@gmail.com) wrote: > I have a need to update my mappings and will therefore need to reindex - but > will be doing so directly from an existing ES index (I'm using ES as my data > store). I've read other posts, and think the following strategy will work, > but have a couple questions: > > Here's the strategy: > > 1 - create new index with new mappings and different index name > 2 - extract data from old index (e.g., using scroll) > 3 - bulk load from old index into new index > 4 - confirm consistency between indices (how do I do this?) > 5 - add 'alias' to new index to map old index name to new index > 6 - delete old index. > > I have two questions regarding the strategy above: > > 1 - I'm not clear on how to confirm that the new index is fully consistent > with the old index. Any suggestions? Understood that I may need to halt > writes to the old index while the comparison is being performed. > > 2 - There will be a moment (however brief) where both the new index (with > the alias) and old index are getting requests. Once the alias is in place, > will search results against both indices create duplicates in the result set > (i.e., the same doc will show up twice)? If so - it might create problems > in my application. Probably a minimal concern - but I'd like to hear > whether anyone else has had issues "swapping" indices using the alias > functionality in this manner. > > Many thanks in advance for your input. > > -- -- ---
**Author:** ![stratawing](https://avatars.discourse-cdn.com/v4/letter/s/9de0a6/32.png) [@stratawing](https://discuss.elastic.co/u/stratawing)\ **Post date:** [August 29, 2012, 11:10pm UTC](https://discuss.elastic.co/t/reindexing-strategy/8884/3 "2012-08-29T23:10:20Z")
Thanks Ivan! Very helpful answer on question 2. Regarding question 1, I really just want to confirm that all documents from the old index were successfully transferred to the new index. In light of your comments, I believe the best approach is to just do a number of documents/fields check, and to check the response from the bulk action to see if there were write errors during the transfer. Unless I'm missing something, this should get me enough information to confirm that everything was transferred. Let me know if I'm mistaken. Thanks again! On Wednesday, August 29, 2012 6:30:25 PM UTC-4, Ivan Brusic wrote: > For question 1, I am unsure about what your definition of > "consistency" might be. The availability of documents? The number of > documents/fields? The relevancy of certain searches? Changing your > mapping will affect your index and by definition, it will not be the > same as the original one. Consistency in the data world usual means > the consistent propagation of changes. Updating a mapping would not > affect that. > > Question 2: > I use aliases for searching/incremental-updates, real index names for > full indexing. Once a new index is created, you can create an atomic > transaction that will remove the alias from the old index and create > an alias on the new one. Searches only know about the alias, never the > real index name. > > Cheers, > > Ivan > > On Wed, Aug 29, 2012 at 10:54 AM, stratawing \<[strat...@gmail.com](mailto:strat...@gmail.com)\\> > wrote: > > > I have a need to update my mappings and will therefore need to reindex - > > but > > will be doing so directly from an existing ES index (I'm using ES as my > > data > > store). I've read other posts, and think the following strategy will > > work, > > but have a couple questions: > > > > Here's the strategy: > > > > 1 - create new index with new mappings and different index name > > 2 - extract data from old index (e.g., using scroll) > > 3 - bulk load from old index into new index > > 4 - confirm consistency between indices (how do I do this?) > > 5 - add 'alias' to new index to map old index name to new index > > 6 - delete old index. > > > > I have two questions regarding the strategy above: > > > > 1 - I'm not clear on how to confirm that the new index is fully > > consistent > > with the old index. Any suggestions? Understood that I may need to halt > > writes to the old index while the comparison is being performed. > > > > 2 - There will be a moment (however brief) where both the new index > > (with > > the alias) and old index are getting requests. Once the alias is in > > place, > > will search results against both indices create duplicates in the result > > set > > (i.e., the same doc will show up twice)? If so - it might create > > problems > > in my application. Probably a minimal concern - but I'd like to hear > > whether anyone else has had issues "swapping" indices using the alias > > functionality in this manner. > > > > Many thanks in advance for your input. > > > > -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:14am UTC](https://discuss.elastic.co/t/reindexing-strategy/8884/4 "2017-07-06T03:14:42Z")