# Logstash multiple pipeline on docker

**URL:** <https://discuss.elastic.co/t/logstash-multiple-pipeline-on-docker/241301>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [July 15, 2020, 1:16pm UTC](https://discuss.elastic.co/t/logstash-multiple-pipeline-on-docker/241301 "2020-07-15T13:16:45Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Abolurah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abolurah/32/68804_2.png) [@Abolurah](https://discuss.elastic.co/u/Abolurah)\
**Post date:** [July 15, 2020, 1:16pm UTC](https://discuss.elastic.co/t/logstash-multiple-pipeline-on-docker/241301/1 "2020-07-15T13:16:45Z")

</div>

Hi All  
I'm trying to build logstash on docker with multiple pipelines.  
After reading a lot over the internet and trying things, still can't start multiple pipelines on docker.  
I'm trying to run some basic configuration in order to ensure that the mechanism is working , I see that logstash is running , but can't see multiple piplines are working

# pipelines.yml looks like this :

```
- pipeline.id: A11111111

config.string: |
               input {
              generator {
                lines => [
                  "line 1",
                  "line 2",
                  "line 3"
                ]
                count => 1
              }
            }
        output {
           stdout{id => "stdout_plugin111"}
        }

- pipeline.id: B22222222

config.string: |
               input {
              generator {
                lines => [
                  "line 4",
                  "line 5",
                  "line 6"
                ]
                count => 1
              }
            }
        output {
           stdout{id => "stdout_plugin222"}
        }

```

# starting docker from cmd:

`> docker run --rm -it -v /c/logstash/:/usr/share/logstash/pipelines docker.elastic.co/logstash/logstash:7.8.0`

when in "c:\logstash" directory, pipelines.yml file located.

# Output looks like this

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ad19229f7e6fa2037d0904de83ddd5f4873371f.png)

Thanks in advance for your help.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 15, 2020, 5:38pm UTC](https://discuss.elastic.co/t/logstash-multiple-pipeline-on-docker/241301/2 "2020-07-15T17:38:41Z")

</div>

Clearly it is not loading the pipelines.yml that you want it to load. logstash will load pipelines.yml from path.settings

Try enabling log.level debug and see what path.settings is set to.

---

<div class="post-metadata">

**Author:** ![Abolurah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abolurah/32/68804_2.png) [@Abolurah](https://discuss.elastic.co/u/Abolurah)\
**Post date:** [July 16, 2020, 7:56am UTC](https://discuss.elastic.co/t/logstash-multiple-pipeline-on-docker/241301/3 "2020-07-16T07:56:42Z")

</div>

> [@Badger](#):
>
> path.settings

> [2020-07-16T07:49:39,624][DEBUG][logstash.runner] path.settings: "/usr/share/logstash/config"

this is a path.settings, is it OK ?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 16, 2020, 6:04pm UTC](https://discuss.elastic.co/t/logstash-multiple-pipeline-on-docker/241301/4 "2020-07-16T18:04:14Z")

</div>

Is that where pipelines.yml exists? If so, does the use who is running logstash have permissions to the file and directory? Also, if this is docker, are you sure the file exists in the container?

---

<div class="post-metadata">

**Author:** ![Abolurah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abolurah/32/68804_2.png) [@Abolurah](https://discuss.elastic.co/u/Abolurah)\
**Post date:** [July 19, 2020, 9:09am UTC](https://discuss.elastic.co/t/logstash-multiple-pipeline-on-docker/241301/5 "2020-07-19T09:09:05Z")

</div>

@Badger thanks for reply.  
After I placed logstash,yml and pipelines.yml to the same directory (c:\Logstash in my case) and set path config of the second pipeline (as additional file) it worked.

#pipelines.yml

```
- pipeline.id: test1111111111111
  pipeline.workers: 1
  pipeline.batch.size: 1
  config.string: 'input { 
  generator {  
  lines => ["line 4", "line 5", "line 6"]     
  count => 1 } } 
  output { stdout{id => "stdout_plugi3333322"}}'

- pipeline.id: test2222222222222
  pipeline.workers: 1
  path.config: "/usr/share/logstash/config/second.conf"

```

#second.conf in the same directory as pipelines.yml and logstash.yml  
input {  
generator {  
lines =\> ["line 4", "line 5", "line 6"]  
count =\> 1 } }  
output { stdout{id =\> "stdout\_plugin22222"}}

# command line run of logstash in docker

> docker run --rm -it -v /c/logstash/:/usr/share/logstash/config [docker.elastic.co/logstash/logstash:7.8.0](http://docker.elastic.co/logstash/logstash:7.8.0) --log.level=debug

- This is a working example for those who looking to run multiple pipelines of logstash on docker.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 16, 2020, 9:09am UTC](https://discuss.elastic.co/t/logstash-multiple-pipeline-on-docker/241301/6 "2020-08-16T09:09:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.