# Send only one field in pipeline to pipeline communication

**URL:** <https://discuss.elastic.co/t/send-only-one-field-in-pipeline-to-pipeline-communication/308002>\
**Category:** Logstash\
**Created:** [June 23, 2022, 12:03pm UTC](https://discuss.elastic.co/t/send-only-one-field-in-pipeline-to-pipeline-communication/308002 "2022-06-23T12:03:53Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Laetitia\_RICHARD](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/laetitia_richard/32/85029_2.png) [@Laetitia\_RICHARD](https://discuss.elastic.co/u/Laetitia_RICHARD)\
**Post date:** [June 23, 2022, 12:03pm UTC](https://discuss.elastic.co/t/send-only-one-field-in-pipeline-to-pipeline-communication/308002/1 "2022-06-23T12:03:53Z")

</div>

Hello,  
I'd like my 1st pipeline to send the event to an Elasticsearch output and only a field's event to the input of a 2nd pipeline. Is it possible to send only one field in a pipeline and not a complete event?

---

<div class="post-metadata">

**Author:** ![smtp\_server](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smtp_server/32/102266_2.png) [@smtp\_server](https://discuss.elastic.co/u/smtp_server)\
**Post date:** [June 23, 2022, 2:48pm UTC](https://discuss.elastic.co/t/send-only-one-field-in-pipeline-to-pipeline-communication/308002/2 "2022-06-23T14:48:11Z")

</div>

Remove those extra fields using mutate filter.

---

<div class="post-metadata">

**Author:** ![Laetitia\_RICHARD](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/laetitia_richard/32/85029_2.png) [@Laetitia\_RICHARD](https://discuss.elastic.co/u/Laetitia_RICHARD)\
**Post date:** [June 23, 2022, 2:57pm UTC](https://discuss.elastic.co/t/send-only-one-field-in-pipeline-to-pipeline-communication/308002/3 "2022-06-23T14:57:16Z")

</div>

Hello, thank you for your reply. I can't remove them because I need all field for my elasticsearch output.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 23, 2022, 3:01pm UTC](https://discuss.elastic.co/t/send-only-one-field-in-pipeline-to-pipeline-communication/308002/4 "2022-06-23T15:01:01Z")

</div>

The pipeline to pipeline using the `pipeline` output does not support the `codec` option to change the format of the output message, which would allow you to send just one field to a pipeline and the entire message to the other pipeline.

To do this you will need to duplicate your event using the `clone` filter, the `prune` filter and some conditionals to change only the event that was cloned.

Something like this:

```auto
filter {
    clone {
        clones => ["cloned"]
    }
    if [type] == "cloned" {
        prune {
            whitelist_names => ["field-you-want-to-send"]
        }
    }
}
output {
    if [type] == "cloned" {
        pipeline { your pipeline-to-pipeline}
    }
    if [type] != "cloned" {
        your other output
    }
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 23, 2022, 3:13pm UTC](https://discuss.elastic.co/t/send-only-one-field-in-pipeline-to-pipeline-communication/308002/5 "2022-06-23T15:13:19Z")

</div>

If you have two pipelines then just remove the fields (using mutate or prune) in one of them. The other one will send the complete event to elasticsearch. This is the [forked-path pattern](https://www.elastic.co/guide/en/logstash/current/pipeline-to-pipeline.html#forked-path-pattern) for pipeline-to-pipeline communication.

---

<div class="post-metadata">

**Author:** ![Laetitia\_RICHARD](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/laetitia_richard/32/85029_2.png) [@Laetitia\_RICHARD](https://discuss.elastic.co/u/Laetitia_RICHARD)\
**Post date:** [June 23, 2022, 4:18pm UTC](https://discuss.elastic.co/t/send-only-one-field-in-pipeline-to-pipeline-communication/308002/6 "2022-06-23T16:18:31Z")

</div>

Thank you for your answers.

---

<div class="post-metadata">

**Author:** ![Laetitia\_RICHARD](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/laetitia_richard/32/85029_2.png) [@Laetitia\_RICHARD](https://discuss.elastic.co/u/Laetitia_RICHARD)\
**Post date:** [July 4, 2022, 9:32am UTC](https://discuss.elastic.co/t/send-only-one-field-in-pipeline-to-pipeline-communication/308002/7 "2022-07-04T09:32:21Z")

</div>

Hi @Badger, thank you very much for your answer. I need to make sure that my event is passed to the first output (Elasticsearch) before sending it (in a light version, just the identifier) to the next pipeline. If I use the forked path pattern, I won't be able to have this verification, in this case is using clone filter with conditioning outputs the only solution?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 4, 2022, 4:22pm UTC](https://discuss.elastic.co/t/send-only-one-field-in-pipeline-to-pipeline-communication/308002/8 "2022-07-04T16:22:10Z")

</div>

> [@Laetitia\_RICHARD](#):
>
> I need to make sure that my event is passed to the first output (Elasticsearch) before sending it (in a light version, just the identifier) to the next pipeline.

You cannot be certain of the order in which events are written to outputs. You could introduce a sleep filter into the second pipeline but that is a very crude way to address the issue and still not guaranteed to work.

---

<div class="post-metadata">

**Author:** ![Laetitia\_RICHARD](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/laetitia_richard/32/85029_2.png) [@Laetitia\_RICHARD](https://discuss.elastic.co/u/Laetitia_RICHARD)\
**Post date:** [July 5, 2022, 6:43am UTC](https://discuss.elastic.co/t/send-only-one-field-in-pipeline-to-pipeline-communication/308002/9 "2022-07-05T06:43:55Z")

</div>

Thanx. So, would you still advise me to use the forked path ? I have a number of transformation / enrichment filters in the pipeline that indexes in Elastic, so I was concerned that if I made a dedicated pipeline to send the identifier to the second pipeline (with forked path), the processing time would be much faster and therefore my full, enriched event would not have had time to be processed and sent to Elastic. The fact that they were in the same pipeline with two outputs ensured that the event was processed (with all the filters) and would be sent to Elasticsearch at almost the same time.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 5, 2022, 2:17pm UTC](https://discuss.elastic.co/t/send-only-one-field-in-pipeline-to-pipeline-communication/308002/10 "2022-07-05T14:17:08Z")

</div>

Well I guess you could do the fork after the enrichment, and then strip all the enrichment off in the second pipeline. But it feels wrong.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 2, 2022, 2:17pm UTC](https://discuss.elastic.co/t/send-only-one-field-in-pipeline-to-pipeline-communication/308002/11 "2022-08-02T14:17:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
