# How does Logstash handle multiple pipeline files?

**URL:** <https://discuss.elastic.co/t/how-does-logstash-handle-multiple-pipeline-files/111645>\
**Category:** Logstash\
**Created:** [December 13, 2017, 8:46pm UTC](https://discuss.elastic.co/t/how-does-logstash-handle-multiple-pipeline-files/111645 "2017-12-13T20:46:40Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)\
**Post date:** [December 13, 2017, 8:46pm UTC](https://discuss.elastic.co/t/how-does-logstash-handle-multiple-pipeline-files/111645/1 "2017-12-13T20:46:40Z")

</div>

I've tried looking in the documentation and Googling for this, but I couldn't find any answer. I'm unclear about how Logstash handles these. I know there is a `/etc/logstash/conf.d/` directory which you can put your pipeline configuration files in. I'm kind of confused about how this works, especially in relation to the multiple pipelines feature.

- By default, if I add a `my-pipeline.conf` file to `conf.d` then logstash will see it and run it, correct?
- If there are multiple `*.conf` files in `conf.d` then Logstash will concatenate them and process them as one config file with conditional statements, right?
- How does Logstash handle `*.conf` files in `conf.d` if it sees there is a `/etc/logstash/pipelines.yml` file? Will all of the pipelines in `conf.d` still run? Or only the pipelines defined in `pipelines.yml`?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 14, 2017, 8:19am UTC](https://discuss.elastic.co/t/how-does-logstash-handle-multiple-pipeline-files/111645/2 "2017-12-14T08:19:26Z")

</div>

> By default, if I add a `my-pipeline.conf` file to `conf.d` then logstash will see it and run it, correct?

Yes.

> If there are multiple `*.conf` files in `conf.d` then Logstash will concatenate them and process them as one config file with conditional statements, right?

Yes.

> How does Logstash handle `*.conf` files in `conf.d` if it sees there is a `/etc/logstash/pipelines.yml` file? Will all of the pipelines in `conf.d` still run? Or only the pipelines defined in `pipelines.yml`?

Not sure.

---

<div class="post-metadata">

**Author:** ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)\
**Post date:** [December 14, 2017, 4:50pm UTC](https://discuss.elastic.co/t/how-does-logstash-handle-multiple-pipeline-files/111645/3 "2017-12-14T16:50:28Z")

</div>

So for the last question, is it standard practice to not put your pipeline files in `conf.d` when using `pipelines.yml`? If so, where do you normally put them?

---

<div class="post-metadata">

**Author:** ![makibroshett](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/makibroshett/32/106322_2.png) [@makibroshett](https://discuss.elastic.co/u/makibroshett)\
**Post date:** [December 19, 2017, 7:47am UTC](https://discuss.elastic.co/t/how-does-logstash-handle-multiple-pipeline-files/111645/4 "2017-12-19T07:47:37Z")

</div>

When I opened the sample pipelines.yml, I saw that each pipeline can use a path.config property where you store the path of your actual config. So you can still leave your existing configs in /etc/conf.d and reference them in pipelines.yml.  
Or if your configs are really one-liners, you can store them inside pipelines.yml, using the config.string property.

Hope this helps

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 16, 2018, 7:49am UTC](https://discuss.elastic.co/t/how-does-logstash-handle-multiple-pipeline-files/111645/5 "2018-01-16T07:49:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
