# Different index output using ingest pipeline

**URL:** https://discuss.elastic.co/t/different-index-output-using-ingest-pipeline/286325
**Category:** Elasticsearch
**Tags:** ingest-pipeline
**Created:** [October 11, 2021, 3:35am UTC](https://discuss.elastic.co/t/different-index-output-using-ingest-pipeline/286325 "2021-10-11T03:35:54Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![alfianaf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alfianaf/32/93192_2.png) [@alfianaf](https://discuss.elastic.co/u/alfianaf)
#### Post date: [October 11, 2021, 3:35am UTC](https://discuss.elastic.co/t/different-index-output-using-ingest-pipeline/286325/1 "2021-10-11T03:35:54Z")

</div>

Hello, I'm trying to use ingest pipeline to get different index output.  
Here's my case: I have fluentd to hit input on Elasticsearch, fluentd make index input on Elasticsearch named as "app-write", I want to filter from the current source input.  
here is glimpse of my index if I dont use any filter:

```auto
"_source": {
"kubernetes": {
"container_name": "iris-uat"
} }

```

I want to get different index output based on value from [kubernetes][container\_name]. there is only two value on the current field, it is "iris-uat" and "iris".  
Is there any way to get index output as "app-write-iris" and "app-write-iris-uat"?

Keep in mind that I didn't use logstash to filter the index because I'm having much trouble with it (such as overload input or else)

Any help is much appreciated  
Thanks

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 11, 2021, 4:04am UTC](https://discuss.elastic.co/t/different-index-output-using-ingest-pipeline/286325/2 "2021-10-11T04:04:58Z")

</div>

You could do something like this - [Ingest pipelines | Elasticsearch Guide [7.15] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.15/ingest.html#conditionally-apply-pipelines)

---

<div class="post-metadata">

### Author: ![alfianaf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alfianaf/32/93192_2.png) [@alfianaf](https://discuss.elastic.co/u/alfianaf)
#### Post date: [October 11, 2021, 4:11am UTC](https://discuss.elastic.co/t/different-index-output-using-ingest-pipeline/286325/3 "2021-10-11T04:11:35Z")

</div>

I've seen the documentation, but I'm sorry I don't really understand how it will be implemented on my case

```auto
"processors": [
    {
      "pipeline": {
        "description": "If 'service.name' is 'apache_httpd', use 'httpd_pipeline'",
        "if": "ctx.service?.name == 'apache_httpd'",
        "name": "httpd_pipeline"
      }
    }

```

according to the example, can I just change the "name" to index name that I want?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 11, 2021, 4:15am UTC](https://discuss.elastic.co/t/different-index-output-using-ingest-pipeline/286325/4 "2021-10-11T04:15:16Z")

</div>

The `name` is the pipeline you want to use. So you'd need 3 pipelines;

1. The first one which figures out the `container_name` value
2. A pipeline for `iris` that sends to it's own index
3. A pipeline for `iris-uat` that sends to it's own index

---

<div class="post-metadata">

### Author: ![alfianaf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alfianaf/32/93192_2.png) [@alfianaf](https://discuss.elastic.co/u/alfianaf)
#### Post date: [October 11, 2021, 4:50am UTC](https://discuss.elastic.co/t/different-index-output-using-ingest-pipeline/286325/5 "2021-10-11T04:50:36Z")

</div>

from your explanation, I guess this is what I should do

1. Make pipeline named 'iris-rule' that have conditional to get container name  
(if container name 'iris' go to 'iris' pipeline, and do so for 'iris-uat')
2. Make pipeline named 'iris' to just ingest input to 'iris' index
3. Make pipeline named 'iris-uat to just ingest input to 'iris-uat' index

cmiiw, sorry for bothering because I haven't try to make any pipeline before

---

<div class="post-metadata">

### Author: ![alfianaf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alfianaf/32/93192_2.png) [@alfianaf](https://discuss.elastic.co/u/alfianaf)
#### Post date: [October 11, 2021, 5:46am UTC](https://discuss.elastic.co/t/different-index-output-using-ingest-pipeline/286325/6 "2021-10-11T05:46:17Z")

</div>

I've created test pipeline based on your instructions, and It works perfectly, but I tried on reroute api, so I rerouted an index to a new index.  
so for my case, should I state the pipeline rule's name on my index template to apply the pipeline automatically on document input?

Thanks in advance

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 11, 2021, 5:47am UTC](https://discuss.elastic.co/t/different-index-output-using-ingest-pipeline/286325/7 "2021-10-11T05:47:33Z")

</div>

> [@alfianaf](#):
>
> so for my case, should I state the pipeline rule's name on my index template to apply the pipeline automatically on document input?

Ideally, yes.

---

<div class="post-metadata">

### Author: ![alfianaf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alfianaf/32/93192_2.png) [@alfianaf](https://discuss.elastic.co/u/alfianaf)
#### Post date: [October 11, 2021, 5:57am UTC](https://discuss.elastic.co/t/different-index-output-using-ingest-pipeline/286325/8 "2021-10-11T05:57:43Z")

</div>

thanks for the help, hope you have a nice day

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 8, 2021, 5:57am UTC](https://discuss.elastic.co/t/different-index-output-using-ingest-pipeline/286325/9 "2021-11-08T05:57:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
