# Multiple Logstash pipelines

**URL:** <https://discuss.elastic.co/t/multiple-logstash-pipelines/181379>\
**Category:** Logstash\
**Created:** [May 16, 2019, 11:09am UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379 "2019-05-16T11:09:04Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [May 16, 2019, 11:09am UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/1 "2019-05-16T11:09:04Z")

</div>

Hi,

I am using Logstash 6.5 as a service on a windows machine. I have defined a pipelines.yml file that looks like this:  
# List of pipelines to be loaded by Logstash  
- pipeline.id: filebeat  
path.config: "CPA/LogStash/config\logstash-filebeat.conf"

```
# List of pipelines to be loaded by Logstash
 - pipeline.id: K5DLWDBSSIT2909
   path.config: "CPA/LogStash/config/logstash-K5DLWDBSSIT2909.conf"

```

When i run logstash from cmd whitout a parameter like '-f' or '-e' the output is like thiis:  
"Cannot read the pipelines file"

Does anyone have any ideea how can i resolve this?

Thanks!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 16, 2019, 12:22pm UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/2 "2019-05-16T12:22:44Z")

</div>

Is your pipelines.yml file in the directory you have set in path.settings?

---

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [May 16, 2019, 12:34pm UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/3 "2019-05-16T12:34:11Z")

</div>

path.settings is located in logstash.yml?  
Pipelines.yml is located in the same folder with logstash.yml

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 16, 2019, 12:42pm UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/4 "2019-05-16T12:42:25Z")

</div>

Unless you override it on the command line, paths.settings would be set in logstash.yml, yes.

---

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [May 16, 2019, 3:52pm UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/5 "2019-05-16T15:52:40Z")

</div>

The config files that i defined in the path.config must be in the same place where logstash is installed? I don't override in the command line and is still doesn't work..

---

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [May 16, 2019, 3:58pm UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/6 "2019-05-16T15:58:04Z")

</div>

The error is now "Failed to read pipeline yaml file" It matters on which drive is logstash installed? Does it have to be C drive?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 16, 2019, 4:02pm UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/7 "2019-05-16T16:02:21Z")

</div>

Can you post the entire output from logstash when you run it from the command line. Please post it as text, select it, and click on \</\> in the toolbar above the edit panel. You should then see it displayed in the preview panel on the right like this

```
...
```

---

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [May 16, 2019, 4:05pm UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/8 "2019-05-16T16:05:39Z")

</div>

C:\Users\zDL-ChERP\_GSTS\_setup\>E:\TotalSoft\CPA\LogStash\bin\logstash.bat  
Sending Logstash logs to K:/LogStash/Log which is now configured via log4j2.prop  
erties  
ERROR: Failed to read pipelines yaml file. Location: E:/TotalSoft/CPA/LogStash/c  
onfig/pipelines.yml  
usage:  
bin/logstash -f CONFIG\_PATH [-t] [-r] [-w COUNT] [-l LOG]  
bin/logstash --modules MODULE\_NAME [-M "MODULE\_NAME.var.PLUGIN\_TYPE.PLUGIN\_NAM  
E.VARIABLE\_NAME=VALUE"] [-t] [-w COUNT] [-l LOG]  
bin/logstash -e CONFIG\_STR [-t] [--log.level fatal|error|warn|info|debug|trace  
] [-w COUNT] [-l LOG]  
bin/logstash -i SHELL [--log.level fatal|error|warn|info|debug|trace]  
bin/logstash -V [--log.level fatal|error|warn|info|debug|trace]  
bin/logstash --help  
[2019-05-16T18:04:17,952][ERROR][org.logstash.Logstash] java.lang.IllegalSta  
teException: Logstash stopped processing because of an error: (SystemExit) exit

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 16, 2019, 4:09pm UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/9 "2019-05-16T16:09:51Z")

</div>

> [@Costi](#):
>
> Location: E:/TotalSoft/CPA/LogStash/c  
> onfig/pipelines.yml

Does that file exist?

---

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [May 17, 2019, 7:17am UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/10 "2019-05-17T07:17:28Z")

</div>

Yes!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 17, 2019, 11:56am UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/11 "2019-05-17T11:56:44Z")

</div>

If you cd to the E: drive and run the same command do you get the same error?

---

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [May 21, 2019, 5:51am UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/12 "2019-05-21T05:51:09Z")

</div>

Yes, i get the same error. I don't know what else to try.

---

<div class="post-metadata">

**Author:** ![pguinet](https://avatars.discourse-cdn.com/v4/letter/p/e5b9ba/32.png) [@pguinet](https://discuss.elastic.co/u/pguinet)\
**Post date:** [May 24, 2019, 6:54am UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/13 "2019-05-24T06:54:48Z")

</div>

Since it seems to be on Windows, put backslashes "" and not slashes ? (just guessing, never used Elasticsearch on Windows).

---

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [May 26, 2019, 4:43pm UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/14 "2019-05-26T16:43:04Z")

</div>

Hi,

Now i get a different error : error logstash.config.sourceloader no configuration found in the configured sources.

Any help?  
Thanks!

---

<div class="post-metadata">

**Author:** ![pguinet](https://avatars.discourse-cdn.com/v4/letter/p/e5b9ba/32.png) [@pguinet](https://discuss.elastic.co/u/pguinet)\
**Post date:** [May 27, 2019, 5:59am UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/15 "2019-05-27T05:59:16Z")

</div>

You changed slashes to backslashes inside the pipelines.yml too ? ( ```  
path.config: "CPA/LogStash/config/logstash-K5DLWDBSSIT2909.conf"

```auto

```

---

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [May 28, 2019, 11:27am UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/16 "2019-05-28T11:27:20Z")

</div>

Hi,

I try with slashes and with backslashes and it still doesn't work...I don't know what else to try.  
If i run just a conf file without using the pipelines.yml it's all good.

Any suggestions?  
Thanks!

---

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [May 29, 2019, 1:43pm UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/17 "2019-05-29T13:43:22Z")

</div>

SOLVED!!

I moved the config files into a directory conf.d and after that logstash began to read from pipelines.yml

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 26, 2019, 1:43pm UTC](https://discuss.elastic.co/t/multiple-logstash-pipelines/181379/18 "2019-06-26T13:43:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
