# Ingest pipeline not working for filebeat

**URL:** <https://discuss.elastic.co/t/ingest-pipeline-not-working-for-filebeat/228015>\
**Category:** Elasticsearch\
**Created:** [April 15, 2020, 2:29am UTC](https://discuss.elastic.co/t/ingest-pipeline-not-working-for-filebeat/228015 "2020-04-15T02:29:58Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![seanziee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/seanziee/32/45151_2.png) [@seanziee](https://discuss.elastic.co/u/seanziee)\
**Post date:** [April 15, 2020, 2:29am UTC](https://discuss.elastic.co/t/ingest-pipeline-not-working-for-filebeat/228015/1 "2020-04-15T02:29:58Z")

</div>

I've created an ingest pipeline that works perfectly when I use the simulate endpoint, `POST filebeat-7.6.2-2020.04.14/_doc?pipeline=pipeline_lookup_test`, and when I reindex the data into the index. But when I try and send the data from filebeat, the data comes in, but the pipeline doesn't work. I've tried making the pipeline more simple (just setting a field) and also that doesn't work.

I've `PUT filebeat-7.6.2-2020.04.14/_settings { "index" : { "default_pipeline" : "pipeline_lookup_test" } }`  
And also put in filebeat.yml

```auto
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["http://localhost:9200"]
  pipeline: "pipeline_lookup_test"

```

But none of it works. Again, this works when I POST, simulate and reindex, but not when I send the data via filebeat. I'm using a module (zeek), not sure if that would interfere...

```auto
- module: zeek
  # All logs
  connection:
    enabled: true
    var.paths: ["/usr/local/zeek/logs/current/conn*.log"]
  dns:
    enabled: true
    var.paths: ["/usr/local/zeek/logs/current/dns.log", "/usr/local/zeek/logs/current/ntp.log"]
  http:
    enabled: true
    var.paths: ["/usr/local/zeek/logs/current/http.log"]
  files:
    enabled: true
    var.paths: ["/usr/local/zeek/logs/current/files.log"]
  ssl:
    enabled: true
    var.paths: ["/usr/local/zeek/logs/current/ssl.log"]
  notice:
    enabled: true
    var.paths: ["/usr/local/zeek/logs/current/notice.log"]
  weird:
    enabled: true
    var.paths: ["/usr/local/zeek/logs/current/weird.log"]

```

---

<div class="post-metadata">

**Author:** ![seanziee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/seanziee/32/45151_2.png) [@seanziee](https://discuss.elastic.co/u/seanziee)\
**Post date:** [April 15, 2020, 2:40am UTC](https://discuss.elastic.co/t/ingest-pipeline-not-working-for-filebeat/228015/2 "2020-04-15T02:40:36Z")

</div>

Modules run with built in ingest pipelines and you cannot define multiple ingest pipelines. Therefore `PUT filebeat-7.6.2-2020.04.14/_settings { "index" : { "default_pipeline" : "pipeline_lookup_test" } }` should actually be `PUT filebeat-7.6.2-2020.04.14/_settings { "index" : { "final_pipeline" : "pipeline_lookup_test" } }` ... that was a frustrating 3 hours. 😓

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 13, 2020, 2:46am UTC](https://discuss.elastic.co/t/ingest-pipeline-not-working-for-filebeat/228015/3 "2020-05-13T02:46:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.