# Default ingest pipeline not working

**URL:** <https://discuss.elastic.co/t/default-ingest-pipeline-not-working/247639>\
**Category:** Beats\
**Tags:** beats-module, filebeat\
**Created:** [September 5, 2020, 3:44pm UTC](https://discuss.elastic.co/t/default-ingest-pipeline-not-working/247639 "2020-09-05T15:44:09Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![borna\_talebi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/borna_talebi/32/76317_2.png) [@borna\_talebi](https://discuss.elastic.co/u/borna_talebi)\
**Post date:** [September 5, 2020, 3:44pm UTC](https://discuss.elastic.co/t/default-ingest-pipeline-not-working/247639/1 "2020-09-05T15:44:09Z")

</div>

Hi, I'm pretty new in elastic search so sorry if it's a newbie question.  
I want to change the ingest pipeline and index for my cisco ios module. I tried changing them in filebeat.yml file. my config so far:

```auto
filebeat.inputs:
- type: log
  enabled: false
  paths:
    - /var/log/*.log
    #- c:\programdata\elasticsearch\logs\*
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
  #reload.period: 10s
setup.template.settings:
  index.number_of_shards: 1
setup.kibana:
  host: "kibana:5601"
output.elasticsearch:
  hosts: ["node-1:9200"]
  indices:
    - index: "cisco-beat-%{+yyyy.MM}"
      when.contains:
        event.module: "cisco"
  protocol: "https"
  ssl.certificate: "/etc/filebeat/filebeat/filebeat.crt"
  ssl.key: "/etc/filebeat/filebeat/filebeat.key"
  ssl.certificate_authorities:
    - /etc/filebeat/ca/ca.crt
  username: "elastic"
  password: "xxxxxxxxxxxxxxxxx"
  pipeline: geoip-info
  pipelines:
    - pipeline: "ciscosbeatspipeline"
      when.contains:
        event.module: "cisco"
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

```

I created my ingest pipeline (very simple) via dev console :

```auto
 PUT _ingest/pipeline/ciscosbeatspipeline
{
    "description": "cisco ios ingest pipeline",
    "processors": [
        {
            "dissect": {
            "field": "log.original",
            "pattern" : "%{}: %{DeviceName}:"
           }
        },
           {
            "set": {
                "field": "host.name",
                "value": "{{DeviceName}}"
              }
        },
        {
            "remove": {
                "field": "DeviceName"
  }
}
    ]

}

```

Then I changed the default pipeline setting for my index. here is my current index setting:

```auto
GET /cisco-beat-*/_settings
{
  "cisco-beat-2020.09" : {
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "provided_name" : "cisco-beat-2020.09",
        "default_pipeline" : "ciscosbeatspipeline",
        "creation_date" : "1599297439325",
        "number_of_replicas" : "1",
        "uuid" : "xuLvwqnLTHiZIQ6yKICRaQ",
        "version" : {
          "created" : "7080099"
        }
      }
    }
  }
}

```

But my data isn't processing in the ingest pipeline. I checked my pipeline with test pipeline in stack management -\> ingest node management and it works fine(host.name field changes).  
I'm running a single node cluster ELK 7.8.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [September 6, 2020, 5:18am UTC](https://discuss.elastic.co/t/default-ingest-pipeline-not-working/247639/2 "2020-09-06T05:18:08Z")

</div>

> [@borna\_talebi](#):
>
> ```auto
> filebeat.inputs:
> - type: log
> enabled: false
> 
> ```

Did you change `enabled` from `false` to `true`?

Your filebeat input will not work until `enabled` is set to `true`.

---

<div class="post-metadata">

**Author:** ![borna\_talebi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/borna_talebi/32/76317_2.png) [@borna\_talebi](https://discuss.elastic.co/u/borna_talebi)\
**Post date:** [September 6, 2020, 5:32am UTC](https://discuss.elastic.co/t/default-ingest-pipeline-not-working/247639/3 "2020-09-06T05:32:06Z")

</div>

> Did you change `enabled` from `false` to `true` ?
> 
> Your filebeat input will not work until `enabled` is set to `true` .

I don't think that's the problem. filebeat can receive my cisco logs and send it to elasticsearch with the correct index (`"cisco-beat-%{+yyyy.MM}"`) but can't send it to the pipeline

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [September 6, 2020, 5:57am UTC](https://discuss.elastic.co/t/default-ingest-pipeline-not-working/247639/4 "2020-09-06T05:57:10Z")

</div>

Yeah, you are right, you are using the `cisco` module, sorry, saw it later. The `enabled` in the `input` does not affect your case.

So, can you share what are you receiving in Elasticsearch, what fields does your document have?

I'm not sure if filebeat or the cisco module ouputs anything in the field `log.original`. Share a sample document of the what is being indexed.

---

<div class="post-metadata">

**Author:** ![borna\_talebi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/borna_talebi/32/76317_2.png) [@borna\_talebi](https://discuss.elastic.co/u/borna_talebi)\
**Post date:** [September 6, 2020, 6:08am UTC](https://discuss.elastic.co/t/default-ingest-pipeline-not-working/247639/5 "2020-09-06T06:08:48Z")

</div>

[here](https://gist.github.com/bornatalebi/6d73d7139dbddc078e191392f107c3c2) is a sample document.  
also, I created a very simple pipeline (just a set processor) and I still didn't get results. so I don't think there is a problem with ingest pipeline itself.

---

<div class="post-metadata">

**Author:** ![borna\_talebi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/borna_talebi/32/76317_2.png) [@borna\_talebi](https://discuss.elastic.co/u/borna_talebi)\
**Post date:** [September 6, 2020, 6:26am UTC](https://discuss.elastic.co/t/default-ingest-pipeline-not-working/247639/6 "2020-09-06T06:26:03Z")

</div>

After reading [this post](https://discuss.elastic.co/t/ingest-pipeline-not-working-for-filebeat/228015) i changed `final_pipeline` instead of `default_pipeline` and it worked!  
looks like `default_pipeline` doesn't work if your docs are directly coming from filebeat and it's not a re-index or a watcher index action. hope they add a warning in documents.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 4, 2020, 8:26am UTC](https://discuss.elastic.co/t/default-ingest-pipeline-not-working/247639/7 "2020-10-04T08:26:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.