# Configuring Multiple Pipelines

**URL:** <https://discuss.elastic.co/t/configuring-multiple-pipelines/118299>\
**Category:** Logstash\
**Created:** [February 3, 2018, 8:15am UTC](https://discuss.elastic.co/t/configuring-multiple-pipelines/118299 "2018-02-03T08:15:41Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)\
**Post date:** [February 3, 2018, 8:15am UTC](https://discuss.elastic.co/t/configuring-multiple-pipelines/118299/1 "2018-02-03T08:15:41Z")

</div>

Logstash 6.1.1 on Server 2012 R2

Trying to configure multiple pipelines but coming up short. In pipelines.yml:

```
 - pipeline.id: Beats_Pipe
   path.config: "c:/Logstash/config/beats.yml"

```

Here's my beats.yml pipeline:

```
input {
  beats {
    port => 5044
    include_codec_tag => "false"
    id => "Beats_Input"
  }
}
output {
  elasticsearch {
    hosts => "ElasticSearch:9200"
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
  }
}

```

Debug when running bin/logstash (Omitted default LogStash Settings due to post character limitation):

```
[2018-02-03T02:08:54,892][DEBUG][logstash.runner] -------- Logstash Settings (* means modified) ---------
[2018-02-03T02:08:54,892][DEBUG][logstash.runner] *log.level: "trace" (default: "info")
[2018-02-03T02:08:54,892][DEBUG][logstash.runner] *http.host: "192.168.1.101" (default: "127.0.0.1")
[2018-02-03T02:08:54,892][DEBUG][logstash.runner] *http.port: 9600..9600 (default: 9600..9700)
[2018-02-03T02:08:54,892][DEBUG][logstash.runner] *queue.type: "persisted" (default: "memory")
[2018-02-03T02:08:54,892][DEBUG][logstash.runner] *queue.page_capacity: 134217728 (default: 262144000)
[2018-02-03T02:08:54,892][DEBUG][logstash.runner] *queue.max_bytes: 2147483648 (default: 1073741824)
[2018-02-03T02:08:54,907][DEBUG][logstash.runner] *path.queue: "C:/Buffer" (default: "C:/Logstash/data/queue")
[2018-02-03T02:08:54,907][DEBUG][logstash.runner] path.dead_letter_queue: "C:/Logstash/data/dead_letter_queue"
[2018-02-03T02:08:54,907][DEBUG][logstash.runner] --------------- Logstash Settings -------------------
[2018-02-03T02:08:54,938][DEBUG][logstash.config.source.multilocal] Reading pipeline configurations from YAML {:location=>"C:/Logstash/config/pipelines.yml"}
[2018-02-03T02:08:55,079][DEBUG][logstash.agent] Agent: Configuring metric collection
[2018-02-03T02:08:55,142][DEBUG][logstash.instrument.periodicpoller.os] PeriodicPoller: Starting {:polling_interval=>5, :polling_timeout=>120}
[2018-02-03T02:08:55,329][DEBUG][logstash.instrument.periodicpoller.jvm] PeriodicPoller: Starting {:polling_interval=>5, :polling_timeout=>120}
[2018-02-03T02:08:55,532][DEBUG][logstash.instrument.periodicpoller.persistentqueue] PeriodicPoller: Starting {:polling_interval=>5, :polling_timeout=>120}
[2018-02-03T02:08:55,563][DEBUG][logstash.instrument.periodicpoller.deadletterqueue] PeriodicPoller: Starting {:polling_interval=>5, :polling_timeout=>120}
[2018-02-03T02:08:55,673][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"6.1.1"}
[2018-02-03T02:08:55,704][DEBUG][logstash.agent] starting agent
[2018-02-03T02:08:55,767][DEBUG][logstash.agent] Starting puma
[2018-02-03T02:08:55,798][DEBUG][logstash.agent] Trying to start WebServer {:port=>9600}
[2018-02-03T02:08:55,829][DEBUG][logstash.config.source.multilocal] Reading pipeline configurations from YAML {:location=>"C:/Logstash/config/pipelines.yml"}
[2018-02-03T02:08:56,267][DEBUG][logstash.api.service] [api-service] start
[ERROR] 2018-02-03 02:08:56.345 [Ruby-0-Thread-1: C:\Logstash\lib\bootstrap\environment.rb:6] sourceloader - No configuration found in the configured sources.
[2018-02-03T02:08:56,376][DEBUG][logstash.agent] Converging pipelines
[2018-02-03T02:08:56,407][TRACE][logstash.agent] Converge results {:success=>true, :failed_actions=>[], :successful_actions=>[]}
[2018-02-03T02:08:56,438][DEBUG][logstash.instrument.periodicpoller.os] PeriodicPoller: Stopping
[2018-02-03T02:08:56,454][DEBUG][logstash.instrument.periodicpoller.jvm] PeriodicPoller: Stopping
[2018-02-03T02:08:56,454][DEBUG][logstash.instrument.periodicpoller.persistentqueue] PeriodicPoller: Stopping
[2018-02-03T02:08:56,470][DEBUG][logstash.instrument.periodicpoller.deadletterqueue] PeriodicPoller: Stopping
[2018-02-03T02:08:56,548][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2018-02-03T02:09:01,737][DEBUG][logstash.agent] Shutting down all pipelines {:pipelines_count=>0}
[2018-02-03T02:09:01,737][DEBUG][logstash.agent] Converging pipelines
[2018-02-03T02:09:01,753][TRACE][logstash.agent] Converge results {:success=>true, :failed_actions=>[], :successful_actions=>[]}

```

If I run bin/logstash -f and specify the beats.yml pipeline, everything works properly. What am I doing wrong here?

---

<div class="post-metadata">

**Author:** ![mbarclay](https://avatars.discourse-cdn.com/v4/letter/m/34f0e0/32.png) [@mbarclay](https://discuss.elastic.co/u/mbarclay)\
**Post date:** [February 27, 2018, 1:32am UTC](https://discuss.elastic.co/t/configuring-multiple-pipelines/118299/2 "2018-02-27T01:32:30Z")

</div>

Experiencing exactly the same issue, with the same version of Logstash as you. Did you manage to resolve this @wwalker?

---

<div class="post-metadata">

**Author:** ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)\
**Post date:** [February 27, 2018, 1:46am UTC](https://discuss.elastic.co/t/configuring-multiple-pipelines/118299/3 "2018-02-27T01:46:58Z")

</div>

Ya...accidently created two topics on this and answered it in the other thread. In the path.config, remove the drive letter. For example, instead of D:/logstash/config, make it /logstash/config. Yes, that's for Windows.

---

<div class="post-metadata">

**Author:** ![mbarclay](https://avatars.discourse-cdn.com/v4/letter/m/34f0e0/32.png) [@mbarclay](https://discuss.elastic.co/u/mbarclay)\
**Post date:** [February 27, 2018, 8:22pm UTC](https://discuss.elastic.co/t/configuring-multiple-pipelines/118299/4 "2018-02-27T20:22:21Z")

</div>

Great. Thanks very much @wwalker, that worked a treat.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 27, 2018, 8:22pm UTC](https://discuss.elastic.co/t/configuring-multiple-pipelines/118299/5 "2018-03-27T20:22:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.