# Sending request to one index, writing to multiple indices **URL:** \ **Category:** Elasticsearch\ **Created:** [July 11, 2023, 8:50am UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079 "2023-07-11T08:50:43Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Aditya\_Teltia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aditya_teltia/32/122447_2.png) [@Aditya\_Teltia](https://discuss.elastic.co/u/Aditya_Teltia)\ **Post date:** [July 11, 2023, 8:50am UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/1 "2023-07-11T08:50:43Z")
I have a index named `index1`. I want to configure it such that any write/update request that comes to `index1` gets written to both `index1` and `index2` but any search request still uses `index1`. Is this possible with some existing settings in Elasticsearch 7.17.10 or do I have to implement it inside the Elasticsearch? If yes, How can I go by implementing this ? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 11, 2023, 8:51am UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/2 "2023-07-11T08:51:48Z")
That is something I believe you need to implement outside of Elasticsearch. ---
**Author:** ![Aditya\_Teltia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aditya_teltia/32/122447_2.png) [@Aditya\_Teltia](https://discuss.elastic.co/u/Aditya_Teltia)\ **Post date:** [July 11, 2023, 10:22am UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/3 "2023-07-11T10:22:43Z")
Can you suggest some way to achieve this ? I am having hard time finding anything to implement the same. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 11, 2023, 10:26am UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/4 "2023-07-11T10:26:35Z")
It is not possible within Elasticsearch, so you may need to create a proxy of some kind to intercept and duplicate requests. ---
**Author:** ![Aditya\_Teltia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aditya_teltia/32/122447_2.png) [@Aditya\_Teltia](https://discuss.elastic.co/u/Aditya_Teltia)\ **Post date:** [July 11, 2023, 10:37am UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/5 "2023-07-11T10:37:00Z")
Is it possible to implement this inside Elasticsearch maybe by creating a new pipeline processor something like this: ```auto { "processors": [ { "set": { "field": "_index", "value": [index1 , index2] } } ] } ``` Or will this result in some errors ? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 11, 2023, 10:38am UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/6 "2023-07-11T10:38:56Z")
No, I do not think you can do this within Elasticsearch. ---
**Author:** ![Aditya\_Teltia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aditya_teltia/32/122447_2.png) [@Aditya\_Teltia](https://discuss.elastic.co/u/Aditya_Teltia)\ **Post date:** [July 11, 2023, 10:44am UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/7 "2023-07-11T10:44:14Z")
May I know why Elasticsearch doesn't allow to do something like this? Will this result in some sort of error conditions ? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 11, 2023, 11:10am UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/8 "2023-07-11T11:10:28Z")
I do not know why but suspect it could cause issues where a single index request could be partially successful, which is something that is currently not possible as far as I know. I also guess it could have security implications. ---
**Author:** ![Aditya\_Teltia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aditya_teltia/32/122447_2.png) [@Aditya\_Teltia](https://discuss.elastic.co/u/Aditya_Teltia)\ **Post date:** [July 11, 2023, 11:39am UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/9 "2023-07-11T11:39:43Z")
But I am trying to write to an existing index `index1` and a new index `index2`. I don't think this can possibly result in having security implications. I want to know can I add this feature inside Elasticsearch Github codebase. Not saying using any existing feature. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 11, 2023, 11:41am UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/10 "2023-07-11T11:41:40Z")
I do not know if this would at all be possible or whether it would break something, so will leave that for others. ---
**Author:** ![btsinfo](https://avatars.discourse-cdn.com/v4/letter/b/ecccb3/32.png) [@btsinfo](https://discuss.elastic.co/u/btsinfo)\ **Post date:** [July 11, 2023, 12:42pm UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/11 "2023-07-11T12:42:26Z")
To my knowledge, no, there is no specific configuration required to handle your use case. However, you can use a transform to write each entry from Index 1 to Index 2. Please note that this approach does not handle data updates. On the other hand, you can use Logstash to perform such processing and manage additions and updates between the two indexes based on the document\_id. This approach is suitable for a moderate volume of source data but may not be efficient for a very large volume of source data. I believe it would be helpful if you could share your specific business use case to propose a solution more effectively. ---
**Author:** ![Aditya\_Teltia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aditya_teltia/32/122447_2.png) [@Aditya\_Teltia](https://discuss.elastic.co/u/Aditya_Teltia)\ **Post date:** [July 11, 2023, 12:45pm UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/12 "2023-07-11T12:45:37Z")
I want to perform it for moderate volume of data only. While `index1` is going under snapshot and restore only till then I want all the data updates or additions to be written to both the indices `index1` and `index2`. How can I do it using logstash ? Or is there any other way to do the same ? > [@btsinfo](#): > > However, you can use a transform to write each entry from Index 1 to Index 2. Please note that this approach does not handle data updates. Can you describe this approach a little bit more in detail. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [July 11, 2023, 12:49pm UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/13 "2023-07-11T12:49:10Z")
> [@Aditya\_Teltia](#): > > But I am trying to write to an existing index `index1` and a new index `index2`. I don't think this can possibly result in having security implications. I want to know can I add this feature inside Elasticsearch Github codebase. Not saying using any existing feature. This is not possible by design, it would add a lot of complexity and can cause multiple issues, I don't think a feature request to add this would be considered. There are performance issue, management issue, security issues and probably a lot more. What you want to do can be easily done outside Elasticsearch, but you would need to change how you index your data. If you use Logstash you can have two Elasticsearch outputs, each one pointing to one of the indices and for the search you could use an alias, that would only point to one of the indices. > [@Aditya\_Teltia](#): > > While `index1` is going under snapshot and restore only till then I want all the data updates or additions to be written to both the indices `index1` and `index2`. Can you provide more context on why you would want to do that? It is not clear. ---
**Author:** ![Aditya\_Teltia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aditya_teltia/32/122447_2.png) [@Aditya\_Teltia](https://discuss.elastic.co/u/Aditya_Teltia)\ **Post date:** [July 11, 2023, 12:57pm UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/14 "2023-07-11T12:57:09Z")
> [@leandrojmp](#): > > Can you provide more context on why you would want to do that? It is not clear. I want to segregate the updates and addition that happens during the process while still being able to query in `index1` with no data inconsistency. ---
**Author:** ![btsinfo](https://avatars.discourse-cdn.com/v4/letter/b/ecccb3/32.png) [@btsinfo](https://discuss.elastic.co/u/btsinfo)\ **Post date:** [July 11, 2023, 1:05pm UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/15 "2023-07-11T13:05:04Z")
You can use a transform of type 'latest' to achieve the desired behavior where each document in Index 1 is written to Index 2 after a configurable duration, for example, 60 seconds. However, it is important to configure the unique keys and sort fields correctly to meet your requirements ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/c/4c1bde09d9cb0313e8c2fb364ebddf34ed3c7437.png) ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [July 11, 2023, 1:05pm UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/16 "2023-07-11T13:05:19Z")
> [@Aditya\_Teltia](#): > > I want to segregate the updates and addition that happens during the process while still being able to query in `index1` with no data inconsistency. And how writing to two indices would help with that? Assume that you are using Logstash to write data to both `index_1` and `index_2`, and want to create a snapshot of `index_1`, everything written after your snapshot request will be added to both `index_1` and `index_2`, no matter what indice you query it will return the same data. ---
**Author:** ![Aditya\_Teltia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aditya_teltia/32/122447_2.png) [@Aditya\_Teltia](https://discuss.elastic.co/u/Aditya_Teltia)\ **Post date:** [July 11, 2023, 1:09pm UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/17 "2023-07-11T13:09:35Z")
`index_1` initially have huge amount of data which I am transferring to `cluster2` using snapshot and restore. While the process is ongoing. I want those updates and writes segregated from the remaining data. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [July 11, 2023, 1:30pm UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/18 "2023-07-11T13:30:19Z")
If you are writing into two indices, like `index_1` and `index_2`, any document added to `index_1` will also be added to `index_2`, so you will have new writes in the `index_1` that will not be present in the current snapshot. Same thing with updates, unless you update just one of the indices, but again, this will make your data inconsistent between the indices. As already explained in your other [post](https://discuss.elastic.co/t/transferring-a-writable-index-from-one-cluster-to-another/337934/14), I don't think you can achieve what you want without any downtime ---
**Author:** ![Aditya\_Teltia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aditya_teltia/32/122447_2.png) [@Aditya\_Teltia](https://discuss.elastic.co/u/Aditya_Teltia)\ **Post date:** [July 11, 2023, 1:39pm UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/19 "2023-07-11T13:39:40Z")
> [@leandrojmp](#): > > If you are writing into two indices, like `index_1` and `index_2`, any document added to `index_1` will also be added to `index_2`, so you will have new writes in the `index_1` that will not be present in the current snapshot. I am not initially writing to both the indices. I am writing to index\_1 initially then after running snapshot and restore. I want to write to both `index_1` and `index_2` to store the updates and addition segregated and stored in `index_2` while I am still able to query data from `index_1` this will not result in data inconsistency. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 11, 2023, 1:45pm UTC](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079/20 "2023-07-11T13:45:32Z")
> [@Aditya\_Teltia](#): > > this will not result in data inconsistency. If you are performing updates where the existing document is modified instead of overwritten I do not believe this statement is true. Even if you perform updates by overwriting I suspect there would be race conditions where you would have inconsistencies, but it may be less likely. [Next page](https://discuss.elastic.co/t/sending-request-to-one-index-writing-to-multiple-indices/338079.md?page=2)