# Integration Ingest pipeline not executed when logstash ouptut is activated for Agent Policy (Fleet)

**URL:** <https://discuss.elastic.co/t/integration-ingest-pipeline-not-executed-when-logstash-ouptut-is-activated-for-agent-policy-fleet/332936>\
**Category:** Elasticsearch\
**Created:** [May 9, 2023, 1:22pm UTC](https://discuss.elastic.co/t/integration-ingest-pipeline-not-executed-when-logstash-ouptut-is-activated-for-agent-policy-fleet/332936 "2023-05-09T13:22:15Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mehdi-lamrani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mehdi-lamrani/32/108993_2.png) [@mehdi-lamrani](https://discuss.elastic.co/u/mehdi-lamrani)\
**Post date:** [May 9, 2023, 1:22pm UTC](https://discuss.elastic.co/t/integration-ingest-pipeline-not-executed-when-logstash-ouptut-is-activated-for-agent-policy-fleet/332936/1 "2023-05-09T13:22:15Z")

</div>

8.7 here

For some obscure reason,  
when I add a pipeline to an integration via Custom configurations (lower red rectangle in first screen below), it is not triggered when the policy integration output is set to logstash (second screen below).

I need to switch it back to Elasticsearch for the ingest pipeline to kick in.  
Looks like I cant have both Logstash output AND ingest pipeline set this way, for some reason.

Which is a shame because they offer different degrees of granularity (integration level pipelines, policy level pipelines)

PS : I am quasi sure I made both work at the same time at some point but I can't seem to make it work again.

Thanks for your help

 ![Screenshot 2023-05-09 at 13.14.12](https://us1.discourse-cdn.com/elastic/original/3X/5/f/5f626e1c69690d4518d399eaa37aab9db3228561.png)

Screenshot 2023-05-09 at 13.14.12

 ![Screenshot 2023-05-09 at 15.14.08](https://us1.discourse-cdn.com/elastic/original/3X/3/c/3c7263c84c36a005fd4f95670ad6737caecb1eed.png)

---

<div class="post-metadata">

**Author:** ![mehdi-lamrani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mehdi-lamrani/32/108993_2.png) [@mehdi-lamrani](https://discuss.elastic.co/u/mehdi-lamrani)\
**Post date:** [May 9, 2023, 1:33pm UTC](https://discuss.elastic.co/t/integration-ingest-pipeline-not-executed-when-logstash-ouptut-is-activated-for-agent-policy-fleet/332936/2 "2023-05-09T13:33:33Z")

</div>

Do I need to do this ? (Not sure if it makes sense...)

> **[Use ingest pipelines for parsing | Logstash Reference \[8.7\] | Elastic](https://www.elastic.co/guide/en/logstash/current/use-ingest-pipelines.html)**

---

<div class="post-metadata">

**Author:** ![mehdi-lamrani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mehdi-lamrani/32/108993_2.png) [@mehdi-lamrani](https://discuss.elastic.co/u/mehdi-lamrani)\
**Post date:** [May 13, 2023, 12:11pm UTC](https://discuss.elastic.co/t/integration-ingest-pipeline-not-executed-when-logstash-ouptut-is-activated-for-agent-policy-fleet/332936/3 "2023-05-13T12:11:21Z")

</div>

Up ? Anyone ?

Is it possible to use both Logstash output and Ingest pipeline at the same time or not ?

There is no mention that one break the other in the documentation

---

<div class="post-metadata">

**Author:** ![Anton\_H](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anton_h/32/10200_2.png) [@Anton\_H](https://discuss.elastic.co/u/Anton_H)\
**Post date:** [May 13, 2023, 1:12pm UTC](https://discuss.elastic.co/t/integration-ingest-pipeline-not-executed-when-logstash-ouptut-is-activated-for-agent-policy-fleet/332936/4 "2023-05-13T13:12:27Z")

</div>

If you ouput from the agent to logstash then events come from logstash to elasticsearch, I think it was possible to pass pipelines but your case proves otherwise.  
I think it would be best to have the pipeline linked to your index template. That way the source of the events is irrelevant and your events should be processed by the pipeline when they are sent to the index linked to the index template.

edit: are you sure you have configured `pipeline => "%{[@metadata][pipeline]}"` in your logstash ouput?  
If this is what your mentioning, then yes, you need to do that.

```auto
output {
  if [@metadata][pipeline] {
    elasticsearch {
      hosts => "ELASTIC_HOST"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}" 
      action => "create" 
      pipeline => "%{[@metadata][pipeline]}" 
      user => "elastic"
      password => "secret"
    }
  } else {
    elasticsearch {
      hosts => "ELASTIC_HOST""
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}" 
      action => "create"
      user => "elastic"
      password => "secret"
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 10, 2023, 1:12pm UTC](https://discuss.elastic.co/t/integration-ingest-pipeline-not-executed-when-logstash-ouptut-is-activated-for-agent-policy-fleet/332936/5 "2023-06-10T13:12:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.