# Write to multiple indices via one alias **URL:** \ **Category:** Elasticsearch\ **Created:** [May 9, 2012, 4:51pm UTC](https://discuss.elastic.co/t/write-to-multiple-indices-via-one-alias/7620 "2012-05-09T16:51:16Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Nicolas\_Lalevee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nicolas_lalevee/32/2892_2.png) [@Nicolas\_Lalevee](https://discuss.elastic.co/u/Nicolas_Lalevee)\ **Post date:** [May 9, 2012, 4:51pm UTC](https://discuss.elastic.co/t/write-to-multiple-indices-via-one-alias/7620/1 "2012-05-09T16:51:16Z")
It would be useful when I want to do a major change in the mapping. Say I have one index, being used for read and write. Now I create a new index with a new mapping. I want then to have queries still querying the first index, while write go throughout both, and a full reindex goes on the second. When the full reindex is finished, I can then use it for read. And the first index can be deleted. Smooth transition. Currently all of this is managed on the client side, a conf in the db list the read and write indices. For the read, I am changing it so it will be managed via an alias. I was expecting to use aliases for write too but it is not possible the doc says [1]: "It is an error to index to an alias which points to more than one index." I'm a wondering why a such limitation. Nicolas [1] [http://www.elasticsearch.org/guide/reference/api/admin-indices-aliases.html](http://www.elasticsearch.org/guide/reference/api/admin-indices-aliases.html) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [May 9, 2012, 8:03pm UTC](https://discuss.elastic.co/t/write-to-multiple-indices-via-one-alias/7620/2 "2012-05-09T20:03:34Z")
The main reason for the limitation is basically the response, which is a single "index" and not something similar to bulk response. We could simply translates such an index operation to a bulk operation and do it against multiple indices, but then representing the response is problematic... . Open for ideas here, possibly we could have "additional" response section with all the response, or something similar... On Wed, May 9, 2012 at 7:51 PM, Nicolas Lalevée [nicolas.lalevee@hibnet.org](mailto:nicolas.lalevee@hibnet.org)wrote: > It would be useful when I want to do a major change in the mapping. Say I > have one index, being used for read and write. Now I create a new index > with a new mapping. I want then to have queries still querying the first > index, while write go throughout both, and a full reindex goes on the > second. When the full reindex is finished, I can then use it for read. And > the first index can be deleted. Smooth transition. > Currently all of this is managed on the client side, a conf in the db list > the read and write indices. For the read, I am changing it so it will be > managed via an alias. I was expecting to use aliases for write too but it > is not possible the doc says [1]: > "It is an error to index to an alias which points to more than one index." > > I'm a wondering why a such limitation. > > Nicolas > > [1] > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-aliases.html) ---
**Author:** ![Nicolas\_Lalevee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nicolas_lalevee/32/2892_2.png) [@Nicolas\_Lalevee](https://discuss.elastic.co/u/Nicolas_Lalevee)\ **Post date:** [May 10, 2012, 8:38am UTC](https://discuss.elastic.co/t/write-to-multiple-indices-via-one-alias/7620/3 "2012-05-10T08:38:47Z")
Le 9 mai 2012 à 22:03, Shay Banon a écrit : > The main reason for the limitation is basically the response, which is a single "index" and not something similar to bulk response. We could simply translates such an index operation to a bulk operation and do it against multiple indices, but then representing the response is problematic... . Open for ideas here, possibly we could have "additional" response section with all the response, or something similar... Ok I see. Not much idea either, furthermore without breaking the API. Nicolas > On Wed, May 9, 2012 at 7:51 PM, Nicolas Lalevée [nicolas.lalevee@hibnet.org](mailto:nicolas.lalevee@hibnet.org) wrote: > It would be useful when I want to do a major change in the mapping. Say I have one index, being used for read and write. Now I create a new index with a new mapping. I want then to have queries still querying the first index, while write go throughout both, and a full reindex goes on the second. When the full reindex is finished, I can then use it for read. And the first index can be deleted. Smooth transition. > Currently all of this is managed on the client side, a conf in the db list the read and write indices. For the read, I am changing it so it will be managed via an alias. I was expecting to use aliases for write too but it is not possible the doc says [1]: > "It is an error to index to an alias which points to more than one index." > > I'm a wondering why a such limitation. > > Nicolas > > [1] [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-aliases.html) ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [May 10, 2012, 9:06am UTC](https://discuss.elastic.co/t/write-to-multiple-indices-via-one-alias/7620/4 "2012-05-10T09:06:27Z")
On Wed, 2012-05-09 at 23:03 +0300, Shay Banon wrote: > The main reason for the limitation is basically the response, which is > a single "index" and not something similar to bulk response. We could > simply translates such an index operation to a bulk operation and do > it against multiple indices, but then representing the response is > problematic... . Open for ideas here, possibly we could have > "additional" response section with all the response, or something > similar... The typical use case for this is as Nicolas describes: copying all current changes to a new index while copying the old data to the new index. Would a nicer solution not be a river where you could specify the source index and the destination index, and possibly a transformation script (if some data needs to be changed between old and new). You'd probably have to enable the timestamp field to make this possible. clint > On Wed, May 9, 2012 at 7:51 PM, Nicolas Lalevée > [nicolas.lalevee@hibnet.org](mailto:nicolas.lalevee@hibnet.org) wrote: > It would be useful when I want to do a major change in the > mapping. Say I have one index, being used for read and write. > Now I create a new index with a new mapping. I want then to > have queries still querying the first index, while write go > throughout both, and a full reindex goes on the second. When > the full reindex is finished, I can then use it for read. And > the first index can be deleted. Smooth transition. > Currently all of this is managed on the client side, a conf in > the db list the read and write indices. For the read, I am > changing it so it will be managed via an alias. I was > expecting to use aliases for write too but it is not possible > the doc says [1]: > "It is an error to index to an alias which points to more than > one index." > > ``` > I'm a wondering why a such limitation. > > Nicolas > > [1] > http://www.elasticsearch.org/guide/reference/api/admin-indices-aliases.html > > ``` ---
**Author:** ![Jeremie\_BORDIER1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeremie_bordier1/32/2860_2.png) [@Jeremie\_BORDIER1](https://discuss.elastic.co/u/Jeremie_BORDIER1)\ **Post date:** [May 10, 2012, 4:46pm UTC](https://discuss.elastic.co/t/write-to-multiple-indices-via-one-alias/7620/5 "2012-05-10T16:46:51Z")
On Thu, May 10, 2012 at 11:06 AM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com) wrote: > On Wed, 2012-05-09 at 23:03 +0300, Shay Banon wrote: > > > The main reason for the limitation is basically the response, which is > > a single "index" and not something similar to bulk response. We could > > simply translates such an index operation to a bulk operation and do > > it against multiple indices, but then representing the response is > > problematic... . Open for ideas here, possibly we could have > > "additional" response section with all the response, or something > > similar... > > The typical use case for this is as Nicolas describes: copying all > current changes to a new index while copying the old data to the new > index. > > Would a nicer solution not be a river where you could specify the source > index and the destination index, and possibly a transformation script > (if some data needs to be changed between old and new). > > You'd probably have to enable the timestamp field to make this possible. > > clint > > > On Wed, May 9, 2012 at 7:51 PM, Nicolas Lalevée > > [nicolas.lalevee@hibnet.org](mailto:nicolas.lalevee@hibnet.org) wrote: > > It would be useful when I want to do a major change in the > > mapping. Say I have one index, being used for read and write. > > Now I create a new index with a new mapping. I want then to > > have queries still querying the first index, while write go > > throughout both, and a full reindex goes on the second. When > > the full reindex is finished, I can then use it for read. And > > the first index can be deleted. Smooth transition. > > Currently all of this is managed on the client side, a conf in > > the db list the read and write indices. For the read, I am > > changing it so it will be managed via an alias. I was > > expecting to use aliases for write too but it is not possible > > the doc says [1]: > > "It is an error to index to an alias which points to more than > > one index." > > > > ``` > > I'm a wondering why a such limitation. > > > > Nicolas > > > > [1] > > http://www.elasticsearch.org/guide/reference/api/admin-indices-aliases.html > > > > ``` If Elasticsearch had support for collapsing, a possibility would be to use two aliases: - One for searching that spans across the old and new indices - One for indexing that only points to the new index - Collapse on doc type + doc id with higher score on the new index Unfortunately ES doesn't support collapsing so far.. -- Jérémie 'ahFeel' BORDIER ---
**Author:** ![Jack\_Chen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jack_chen/32/2464_2.png) [@Jack\_Chen](https://discuss.elastic.co/u/Jack_Chen)\ **Post date:** [September 6, 2012, 6:20am UTC](https://discuss.elastic.co/t/write-to-multiple-indices-via-one-alias/7620/6 "2012-09-06T06:20:21Z")
+1 for the ability to update multiple indicies via an alias On Friday, 11 May 2012 02:46:51 UTC+10, Jérémie BORDIER wrote: > On Thu, May 10, 2012 at 11:06 AM, Clinton Gormley \<[cl...@traveljury.com](mailto:cl...@traveljury.com)\\> > wrote: > > > On Wed, 2012-05-09 at 23:03 +0300, Shay Banon wrote: > > > > > The main reason for the limitation is basically the response, which is > > > a single "index" and not something similar to bulk response. We could > > > simply translates such an index operation to a bulk operation and do > > > it against multiple indices, but then representing the response is > > > problematic... . Open for ideas here, possibly we could have > > > "additional" response section with all the response, or something > > > similar... > > > > The typical use case for this is as Nicolas describes: copying all > > current changes to a new index while copying the old data to the new > > index. > > > > Would a nicer solution not be a river where you could specify the source > > index and the destination index, and possibly a transformation script > > (if some data needs to be changed between old and new). > > > > You'd probably have to enable the timestamp field to make this possible. > > > > clint > > > > > On Wed, May 9, 2012 at 7:51 PM, Nicolas Lalevée > > > \<[nicolas...@hibnet.org](mailto:nicolas...@hibnet.org) \\> wrote: > > > It would be useful when I want to do a major change in the > > > mapping. Say I have one index, being used for read and write. > > > Now I create a new index with a new mapping. I want then to > > > have queries still querying the first index, while write go > > > throughout both, and a full reindex goes on the second. When > > > the full reindex is finished, I can then use it for read. And > > > the first index can be deleted. Smooth transition. > > > Currently all of this is managed on the client side, a conf in > > > the db list the read and write indices. For the read, I am > > > changing it so it will be managed via an alias. I was > > > expecting to use aliases for write too but it is not possible > > > the doc says [1]: > > > "It is an error to index to an alias which points to more than > > > one index." > > > > > > ``` > > > I'm a wondering why a such limitation. > > > > > > Nicolas > > > > > > [1] > > > > > > ``` > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-aliases.html) > > > > > > If Elasticsearch had support for collapsing, a possibility would be to > use two aliases: > > - One for searching that spans across the old and new indices > - One for indexing that only points to the new index > - Collapse on doc type + doc id with higher score on the new index > > Unfortunately ES doesn't support collapsing so far.. > > -- > Jérémie 'ahFeel' BORDIER -- ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [September 10, 2012, 9:27am UTC](https://discuss.elastic.co/t/write-to-multiple-indices-via-one-alias/7620/7 "2012-09-10T09:27:07Z")
On Wed, 2012-05-09 at 18:51 +0200, Nicolas Lalevée wrote: > It would be useful when I want to do a major change in the mapping. Say I have one index, being used for read and write. Now I create a new index with a new mapping. I want then to have queries still querying the first index, while write go throughout both, and a full reindex goes on the second. When the full reindex is finished, I can then use it for read. And the first index can be deleted. Smooth transition. > Currently all of this is managed on the client side, a conf in the db list the read and write indices. For the read, I am changing it so it will be managed via an alias. I was expecting to use aliases for write too but it is not possible the doc says [1]: > "It is an error to index to an alias which points to more than one index." Are there any other use cases for writing to multiple indices? I ask because the above use case will probably be better handled by the planned changes stream. clint -- ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [September 10, 2012, 3:28pm UTC](https://discuss.elastic.co/t/write-to-multiple-indices-via-one-alias/7620/8 "2012-09-10T15:28:26Z")
I agree, because I think writing to multiple indices is only a special case of a more general one. One exciting use case for a changes stream would be having a client that is fed by pushed "change events" from one index, and transporting them to other targets, possibly to a new index locally or even to another index on a remote cluster, whatever. A backup/restore operation would be required to complement this use case by reading the indexed docs and write them in bulk mode to targets, in case the change stream hook is activated later than the index was created. Not sure about how the scan/scroll operation works, if it serves the docs in the order they were indexed it would be perfect. I have to find out how to handle the condition when the bulk mode should stop and the event mode steps in. Best regards, Jörg On Monday, September 10, 2012 11:27:13 AM UTC+2, Clinton Gormley wrote: > I ask because the above use case will probably be better handled by the > planned changes stream. > > clint -- ---
**Author:** ![Paul\_Smith](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paul_smith/32/1323_2.png) [@Paul\_Smith](https://discuss.elastic.co/u/Paul_Smith)\ **Post date:** [September 11, 2012, 9:49pm UTC](https://discuss.elastic.co/t/write-to-multiple-indices-via-one-alias/7620/9 "2012-09-11T21:49:46Z")
On 11 September 2012 01:28, Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > I agree, because I think writing to multiple indices is only a special > case of a more general one. > > One exciting use case for a changes stream would be having a client that > is fed by pushed "change events" from one index, and transporting them to > other targets, possibly to a new index locally or even to another index on > a remote cluster, whatever. > A backup/restore operation would be required to complement this use case > by reading the indexed docs and write them in bulk mode to targets, in case > the change stream hook is activated later than the index was created. Not > sure about how the scan/scroll operation works, if it serves the docs in > the order they were indexed it would be perfect. I have to find out how to > handle the condition when the bulk mode should stop and the event mode > steps in. This is almost 'transaction log shipping' in a traditional DB model, and I think would be awesome! I just wanted to confirm this 'changes stream' is related to Jörg's Websockets transport plugin? Clint mentioned it as if it was part of the ES roadmap, and I know Shay talked about an inter-DC replication mechanism plan at some point, and I just wanted to confirm that the changes stream concept is what that was about. Maybe this Websockets thing is another way to think of this? When I read about the Websockets plugin and the changes stream, it didn't click about what that meant, but this is quite exciting. I can see how in our case, and maybe others, one could use the dynamic settings to partition a current index to live on a certain # nodes, leaving a set of nodes carved out for the new index (partition the IO load), so that this changes stream can be interleaved with another process that is doing the raw reindex of the original data, allowing live changes to continue to be applied, and allow any VersionConflictException to simply be trapped and ignored as a case of 'ok well the latest information is now already there'. At the end of the process you have a newly created index with all recent changes applied as well, ready for a hot-swap via an alias perhaps. Is that sort of the idea you had in mind Jörg ? cheers, Paul -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:13am UTC](https://discuss.elastic.co/t/write-to-multiple-indices-via-one-alias/7620/10 "2017-07-06T03:13:05Z")