# How to handle searching across all indexes while reindex procedure is running **URL:** \ **Category:** Elasticsearch\ **Created:** [February 18, 2015, 12:23am UTC](https://discuss.elastic.co/t/how-to-handle-searching-across-all-indexes-while-reindex-procedure-is-running/22222 "2015-02-18T00:23:38Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Dave\_Reed](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@Dave\_Reed](https://discuss.elastic.co/u/Dave_Reed)\ **Post date:** [February 18, 2015, 12:23am UTC](https://discuss.elastic.co/t/how-to-handle-searching-across-all-indexes-while-reindex-procedure-is-running/22222/1 "2015-02-18T00:23:38Z")
I have a setup that will cause there to be a few hundred indexes, each of which may need to be rebuilt at any given time. To accomplish this, I'm creating an alias for each index, which point to an incrementing index number. For example, the state of the indexes may look like this: Index -\> Alias indexA -\> indexA\_v1 indexB -\> indexB\_v2 indexC -\> indexC\_v11 indexD -\> indexD\_v3 An index may need to be rebuilt because the mappings need to change. I detect when this happens and go through a reindex procedure: 1. Create a new index (e.g. IndexA\_v2) with the new mappings. 2. Start indexing all the documents into it (could take an hour or two). 3. Swap the "IndexA" alias to point to IndexA\_v2. 4. Delete IndexA\_v1. I need these to be in separate indexes, not just separate types, because the mappings can vary widely and potentially include the same field names but with different mappings. I've learned that mappings should have unique names within an index, even though types can have separate mappings. For searches, there are index-specific searches that will just search on the alias name for the index required (e.g. IndexA). No problem there. But there is type of search in the app that needs to search across all of these indexes. But since there could be hundreds of them, adding them to the search URL seems wrong -- I would think I'd hit some kind of URL limit at some point: [http://server:9200/indexA,indexB,indexC](http://server:9200/indexA,indexB,indexC).....indexZZZ/documenttype/\_search I could use "\_all" instead of listing them individually, but with a specific type that I know only those indexes will have: [http://server:9200/\_all/documenttype/\_search](http://server:9200/_all/documenttype/_search) When there is no reindex in progress, that works great. But... the "\_all" will include results from any indexes that are in the process of being rebuilt, meaning that I could get duplicate results from those indexes during the reindex procedure. I hope that makes sense. I'm open to a different layout if this isn't the ideal way to handle it! Thanks for any help -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7ad24fda-a7ca-4ed8-82ff-70797da1a736%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7ad24fda-a7ca-4ed8-82ff-70797da1a736%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 18, 2015, 3:38am UTC](https://discuss.elastic.co/t/how-to-handle-searching-across-all-indexes-while-reindex-procedure-is-running/22222/2 "2015-02-18T03:38:53Z")
Why not creating another alias "index\_all" on all indices? And update it within step 3? -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 18 févr. 2015 à 01:23, Dave Reed [infinity88@gmail.com](mailto:infinity88@gmail.com) a écrit : > > I have a setup that will cause there to be a few hundred indexes, each of which may need to be rebuilt at any given time. To accomplish this, I'm creating an alias for each index, which point to an incrementing index number. For example, the state of the indexes may look like this: > > Index -\> Alias > indexA -\> indexA\_v1 > indexB -\> indexB\_v2 > indexC -\> indexC\_v11 > indexD -\> indexD\_v3 > > An index may need to be rebuilt because the mappings need to change. I detect when this happens and go through a reindex procedure: > > 1. Create a new index (e.g. IndexA\_v2) with the new mappings. > 2. Start indexing all the documents into it (could take an hour or two). > 3. Swap the "IndexA" alias to point to IndexA\_v2. > 4. Delete IndexA\_v1. > > I need these to be in separate indexes, not just separate types, because the mappings can vary widely and potentially include the same field names but with different mappings. I've learned that mappings should have unique names within an index, even though types can have separate mappings. > > For searches, there are index-specific searches that will just search on the alias name for the index required (e.g. IndexA). No problem there. > > But there is type of search in the app that needs to search across all of these indexes. But since there could be hundreds of them, adding them to the search URL seems wrong -- I would think I'd hit some kind of URL limit at some point: > > [http://server:9200/indexA,indexB,indexC.....indexZZZ/documenttype/\_search](http://server:9200/indexA,indexB,indexC.....indexZZZ/documenttype/_search) > > I could use "\_all" instead of listing them individually, but with a specific type that I know only those indexes will have: > > [http://server:9200/\_all/documenttype/\_search](http://server:9200/_all/documenttype/_search) > > When there is no reindex in progress, that works great. But... the "\_all" will include results from any indexes that are in the process of being rebuilt, meaning that I could get duplicate results from those indexes during the reindex procedure. > > I hope that makes sense. I'm open to a different layout if this isn't the ideal way to handle it! > > Thanks for any help > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7ad24fda-a7ca-4ed8-82ff-70797da1a736%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7ad24fda-a7ca-4ed8-82ff-70797da1a736%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/EEDA5B2E-487B-4EEF-B70F-48731E2634C5%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/EEDA5B2E-487B-4EEF-B70F-48731E2634C5%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Dave\_Reed](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@Dave\_Reed](https://discuss.elastic.co/u/Dave_Reed)\ **Post date:** [February 18, 2015, 4:05am UTC](https://discuss.elastic.co/t/how-to-handle-searching-across-all-indexes-while-reindex-procedure-is-running/22222/3 "2015-02-18T04:05:55Z")
Ah ha, yes indeed 🙂 I knew an index could have multiple aliases, but didn't connect the dots. That will work perfectly 🙂 Thanks! On Tuesday, February 17, 2015 at 7:39:03 PM UTC-8, David Pilato wrote: > Why not creating another alias "index\_all" on all indices? > And update it within step 3? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 18 févr. 2015 à 01:23, Dave Reed \<[infin...@gmail.com](mailto:infin...@gmail.com) \\> a > écrit : > > I have a setup that will cause there to be a few hundred indexes, each of > which may need to be rebuilt at any given time. To accomplish this, I'm > creating an alias for each index, which point to an incrementing index > number. For example, the state of the indexes may look like this: > > Index -\> Alias > indexA -\> indexA\_v1 > indexB -\> indexB\_v2 > indexC -\> indexC\_v11 > indexD -\> indexD\_v3 > > An index may need to be rebuilt because the mappings need to change. I > detect when this happens and go through a reindex procedure: > > 1. Create a new index (e.g. IndexA\_v2) with the new mappings. > 2. Start indexing all the documents into it (could take an hour or two). > 3. Swap the "IndexA" alias to point to IndexA\_v2. > 4. Delete IndexA\_v1. > > I need these to be in separate indexes, not just separate types, because > the mappings can vary widely and potentially include the same field names > but with different mappings. I've learned that mappings should have unique > names within an index, even though types can have separate mappings. > > For searches, there are index-specific searches that will just search on > the alias name for the index required (e.g. IndexA). No problem there. > > But there is type of search in the app that needs to search across all of > these indexes. But since there could be hundreds of them, adding them to > the search URL seems wrong -- I would think I'd hit some kind of URL limit > at some point: > > [http://server:9200/indexA,indexB,indexC.....indexZZZ/documenttype/\_search](http://server:9200/indexA,indexB,indexC.....indexZZZ/documenttype/_search) > > I could use "\_all" instead of listing them individually, but with a > specific type that I know only those indexes will have: > > [http://server:9200/\_all/documenttype/\_search](http://server:9200/_all/documenttype/_search) > > When there is no reindex in progress, that works great. But... the "\_all" > will include results from any indexes that are in the process of being > rebuilt, meaning that I could get duplicate results from those indexes > during the reindex procedure. > > I hope that makes sense. I'm open to a different layout if this isn't the > ideal way to handle it! > > Thanks for any help > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/7ad24fda-a7ca-4ed8-82ff-70797da1a736%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7ad24fda-a7ca-4ed8-82ff-70797da1a736%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/7ad24fda-a7ca-4ed8-82ff-70797da1a736%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/7ad24fda-a7ca-4ed8-82ff-70797da1a736%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/968e8714-e7dd-4c3a-8747-a022a147ca16%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/968e8714-e7dd-4c3a-8747-a022a147ca16%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:32am UTC](https://discuss.elastic.co/t/how-to-handle-searching-across-all-indexes-while-reindex-procedure-is-running/22222/4 "2017-07-06T00:32:02Z")