# Filebeat with customized config and ingest\_pipeline

**URL:** https://discuss.elastic.co/t/filebeat-with-customized-config-and-ingest-pipeline/318111
**Category:** Elastic Cloud on Kubernetes (ECK)
**Created:** [November 3, 2022, 1:26pm UTC](https://discuss.elastic.co/t/filebeat-with-customized-config-and-ingest-pipeline/318111 "2022-11-03T13:26:14Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Tanek21](https://avatars.discourse-cdn.com/v4/letter/t/7feea3/32.png) [@Tanek21](https://discuss.elastic.co/u/Tanek21)
#### Post date: [November 3, 2022, 1:26pm UTC](https://discuss.elastic.co/t/filebeat-with-customized-config-and-ingest-pipeline/318111/1 "2022-11-03T13:26:14Z")

</div>

I've deployed eck cluster (Kibana, elasticsearch, filebeat) and want to pass old logstash filters (logstash.conf) as ingest pipeline in elasticsearch. Fortunately, I am able to do the same with different processors.

when i've tested the ingest pipeline with actual index. It was working smooth as expected. It was generating new fields from the "message" field.  
also didn't faced any syntax issue, while I've configured filebeat config to use "ingest\_pipeline: test-pipeline".

But while I was searching for a specific field which has to be created from the message field through ingest pipeline, it wasn't there.

I don't know how to debug and solve this issue ?? here is filebeat.yaml

```auto
    filebeat.idle_timeout: 10s
    filebeat.spool_size: 1024
    logging.level: info
      - type: log
        paths:
          - /var/log/apache2/*
        fields_under_root: true
        fields:
          doctype: apache_access
......
other conf. logs 
....

    processors:
    - add_kubernetes_metadata:
        host: ${NODE_NAME}
        matchers:
        - logs_path:
            logs_path: "/var/log/containers/"

    output.elasticsearch:
      host: ${NODE_NAME}
      hosts:
      - elasticsearch-test-es-default:9200
      password: ${ELASTICSEARCH_PASSWORD}
      protocol: https
      ingest_pipeline: test-pipeline
      ssl.verification_mode: none
      username: elastic

```

---

<div class="post-metadata">

### Author: ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)
#### Post date: [November 4, 2022, 2:54am UTC](https://discuss.elastic.co/t/filebeat-with-customized-config-and-ingest-pipeline/318111/2 "2022-11-04T02:54:12Z")

</div>

It should be just `pipeline`, [Configure the Elasticsearch output | Filebeat Reference [8.5] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/elasticsearch-output.html#pipeline-option-es)

---

<div class="post-metadata">

### Author: ![Tanek21](https://avatars.discourse-cdn.com/v4/letter/t/7feea3/32.png) [@Tanek21](https://discuss.elastic.co/u/Tanek21)
#### Post date: [November 4, 2022, 9:55am UTC](https://discuss.elastic.co/t/filebeat-with-customized-config-and-ingest-pipeline/318111/3 "2022-11-04T09:55:50Z")

</div>

@legoguy1000 Thank for the quick reply.  
I've tried to add only `pipeline` already before but it throws stream error...  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/b/8bf20a9f2532c1573889cbb246dea47974ee1645.png)

Also while monitoring the logs about filebeat, one more thing i found in logs :

```auto
object mapping for [agent] tried to parse field [agent] as object, but found a concrete value

```

---

<div class="post-metadata">

### Author: ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)
#### Post date: [November 4, 2022, 11:59am UTC](https://discuss.elastic.co/t/filebeat-with-customized-config-and-ingest-pipeline/318111/4 "2022-11-04T11:59:19Z")

</div>

What error?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 2, 2022, 11:59am UTC](https://discuss.elastic.co/t/filebeat-with-customized-config-and-ingest-pipeline/318111/5 "2022-12-02T11:59:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
