Multiple Pipelines using docker instance of logstash 6.2.4

[Karthik_Ramachandran]

Using logstash docker image for 6.2.4, we are trying to run multiple pipelines. But it is not working.

We always see "main" pipeline getting started.

With exact same logstash.yml and pipeline.yml settings, the non-docker instance is working fine.

Any extra settings to be done for docker instance?

[_flo (Florian Koch)]

Hi,

are your configs in the correct location?

See here
https://www.elastic.co/guide/en/logstash/current/docker-config.html

[Karthik_Ramachandran]

Hi flo,
Yes. The configs are all correct. Validated all the binds to container.

We are trying to use multi pipeline mode using docker viz. using pipelines.yml to have individual pipelines instead of one single main pipeline using path.config pointing to pipeline paths.

But, not able to achieve multiple pipeline working. When we use logstash in non-docker mode, multiple pipeline is working fine. docker image doesn't.

Below is the setting

docker run -d --name <>
-v /etc/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml
-v /etc/logstash/config/ssl:/usr/share/logstash/config/ssl
-v /etc/logstash/pipeline:/usr/share/logstash/pipeline
-v /var/log/logstash:/var/log/logstash
-v /etc/logstash/config/pipeline.yml:/usr/share/logstash/config/pipeline.yml docker.elastic.co:443/logstash/logstash:6.2.4

http.host: 0.0.0.0
node.name: "log-2bd62c"
#path.config: /usr/share/logstash/pipeline
dead_letter_queue.enable: true
dead_letter_queue.max_bytes: 1024mb
path.dead_letter_queue: "/var/log/logstash/deadletter"

[Karthik_Ramachandran]

Figured out the cause.

It is pipelines.yml, instead used pipeline.yml.. "s" is the issue. Ah....