# How to set up each pipeline for each index in filebeats?

**URL:** <https://discuss.elastic.co/t/how-to-set-up-each-pipeline-for-each-index-in-filebeats/298319>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 26, 2022, 2:46am UTC](https://discuss.elastic.co/t/how-to-set-up-each-pipeline-for-each-index-in-filebeats/298319 "2022-02-26T02:46:05Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![chenx319](https://avatars.discourse-cdn.com/v4/letter/c/a4c791/32.png) [@chenx319](https://discuss.elastic.co/u/chenx319)\
**Post date:** [February 26, 2022, 2:46am UTC](https://discuss.elastic.co/t/how-to-set-up-each-pipeline-for-each-index-in-filebeats/298319/1 "2022-02-26T02:46:05Z")

</div>

I use one filebeats, to ship two log  
1 hightec.log  
2 ghslog.log

they are different format ,  
I want to set up different pipeline for the each log ,  
can you tell me how to config it ?

Thanks!

```auto
- type : log

  enabled : true

  paths :

    #- /var/log/*.log

- D:\Software\HDP\test_log\hightec.log

  fields :

    type : "hightec"

- type : log

  enabled : true

  paths :

    #- /var/log/*.log

    - D:\Software\HDP\test_log\GHSLOG.log

  fields :

type : "ghs"

 

setup.template.name : "index2-%{[agent.version]}"

setup.template.pattern : "index2-%{[agent.version]}-*"

 

output.elasticsearch :

  # Array of hosts to connect to.

  hosts : ["174.34.93.62:9200"]

  # pipeline: "ne_user_lookup"

  index : "index2-%{[agent.version]}-%{[fields.type]:other}-%{+yyyy.MM.dd}"

 

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 26, 2022, 3:38am UTC](https://discuss.elastic.co/t/how-to-set-up-each-pipeline-for-each-index-in-filebeats/298319/2 "2022-02-26T03:38:25Z")

</div>

You can define a `pipeline` in each input and the appropriate pipeline will be used in the output.

See [here](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html#_pipeline_11)

```auto
- type : log

  enabled : true

  pipeline: my-pipeline
...

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 26, 2022, 5:38am UTC](https://discuss.elastic.co/t/how-to-set-up-each-pipeline-for-each-index-in-filebeats/298319/3 "2022-03-26T05:38:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.