# Use Ingestion Pipeline to split between two indexes **URL:** \ **Category:** Elasticsearch\ **Tags:** ingest-pipeline\ **Created:** [May 4, 2022, 3:18pm UTC](https://discuss.elastic.co/t/use-ingestion-pipeline-to-split-between-two-indexes/303958 "2022-05-04T15:18:12Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![painless\_elastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/painless_elastic/32/94652_2.png) [@painless\_elastic](https://discuss.elastic.co/u/painless_elastic)\ **Post date:** [May 4, 2022, 3:18pm UTC](https://discuss.elastic.co/t/use-ingestion-pipeline-to-split-between-two-indexes/303958/1 "2022-05-04T15:18:12Z")
I have documents containing the field "Status", this can have three values "Draft", "In Progress", or "Approved". I am trying to pass this document through a ingest pipeline, and if the status is equal to "Approved" then it should add it in the **B** index, whereas by default it should index in **A** index irrespective of status value. for ex - 1. ```auto { "id":"123", "status":"Draft" } ``` 1. ```auto { "id":"1234", "status":"InProgress" } ``` 1. ```auto { "id":"12345", "status":"Approved" } ``` **1,2,3 document should go to A Index** and **only document 3 should go to B Index** Is it possible to do it via Ingest Pipeline? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 5, 2022, 4:08am UTC](https://discuss.elastic.co/t/use-ingestion-pipeline-to-split-between-two-indexes/303958/2 "2022-05-05T04:08:12Z")
You should be able to do this with [Pipeline processor | Elasticsearch Guide [8.2] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/pipeline-processor.html), where you push either one to a separate pipeline. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [May 5, 2022, 5:40am UTC](https://discuss.elastic.co/t/use-ingestion-pipeline-to-split-between-two-indexes/303958/3 "2022-05-05T05:40:36Z")
There is actually and example of this [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#conditionally-apply-pipelines) as well ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 5, 2022, 5:59am UTC](https://discuss.elastic.co/t/use-ingestion-pipeline-to-split-between-two-indexes/303958/4 "2022-05-05T05:59:17Z")
Ahh that's what I was looking for as well! ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 5, 2022, 7:09am UTC](https://discuss.elastic.co/t/use-ingestion-pipeline-to-split-between-two-indexes/303958/5 "2022-05-05T07:09:35Z")
You can override the index name through an index pipeline, but I do not believe you can make a document go to more than a single index. ---
**Author:** ![painless\_elastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/painless_elastic/32/94652_2.png) [@painless\_elastic](https://discuss.elastic.co/u/painless_elastic)\ **Post date:** [May 5, 2022, 12:45pm UTC](https://discuss.elastic.co/t/use-ingestion-pipeline-to-split-between-two-indexes/303958/6 "2022-05-05T12:45:31Z")
@stephenb @Christian_Dahlqvist @warkolm Yes, I am able to divert the document to either of the indexes based on condition. But as @Christian_Dahlqvist mentioned, one of the conditions requires pushing the document to both indexes. I tried but looks like it's not possible by this route. Is there any other way to sync to two indexes same time? Does Alias index of these two, will push in both indexes ?? ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [May 5, 2022, 12:53pm UTC](https://discuss.elastic.co/t/use-ingestion-pipeline-to-split-between-two-indexes/303958/7 "2022-05-05T12:53:27Z")
An alias will not write to more than one index at time, so it would not help you. You will need to use logstash to do that, or maybe you can use a [Transform](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) to do that. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 5, 2022, 12:55pm UTC](https://discuss.elastic.co/t/use-ingestion-pipeline-to-split-between-two-indexes/303958/8 "2022-05-05T12:55:38Z")
Why are you looking to duplicate some of the data? ---
**Author:** ![painless\_elastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/painless_elastic/32/94652_2.png) [@painless\_elastic](https://discuss.elastic.co/u/painless_elastic)\ **Post date:** [May 5, 2022, 1:07pm UTC](https://discuss.elastic.co/t/use-ingestion-pipeline-to-split-between-two-indexes/303958/9 "2022-05-05T13:07:20Z")
So, our document has **three stages** , **Draft, in progress, and then Approved** (Ready to be viewed by end consumer). we are thinking that if we put all approved data in a different index, as it will have fewer documents, response time will be faster for our end consumer. As approved instances will be 10 percent of all instances (draft, in progress combined)). Whereas documents having a **draft, in progress state, and Approved will be searched only internally** , so we can have some lag there in response time by putting them in one index. One Suggestion is to query both indexes when a search is internal, it will do the job, but it will also consider the document as a duplicate between the indexes as only one field is different. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [May 5, 2022, 3:34pm UTC](https://discuss.elastic.co/t/use-ingestion-pipeline-to-split-between-two-indexes/303958/10 "2022-05-05T15:34:02Z")
Couple Thoughts How many Documents Total are we talking about .. Term Queries are very fast... If Approved is a value in a `keyword` field which then can be used in a filter contesxt in a query that should be very fast. I think You could also do something like a Latest Transform with a filter on approved to create an index of just the last approved state of a document. > **[Transform overview | Elasticsearch Guide \[8.2\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.2/transform-overview.html#latest-transform-overview)** ![Screen Shot 2022-05-05 at 8.32.57 AM](https://us1.discourse-cdn.com/elastic/original/3X/c/6/c6f0f5be63a036882afa8e93f3ea0f6721b8fe37.png) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 2, 2022, 3:34pm UTC](https://discuss.elastic.co/t/use-ingestion-pipeline-to-split-between-two-indexes/303958/11 "2022-06-02T15:34:33Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.