# Problems With configure Multi Pipelines Logtash V6.1.2

**URL:** <https://discuss.elastic.co/t/problems-with-configure-multi-pipelines-logtash-v6-1-2/121146>\
**Category:** Logstash\
**Created:** [February 22, 2018, 11:56pm UTC](https://discuss.elastic.co/t/problems-with-configure-multi-pipelines-logtash-v6-1-2/121146 "2018-02-22T23:56:42Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![agonex](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@agonex](https://discuss.elastic.co/u/agonex)\
**Post date:** [February 22, 2018, 11:56pm UTC](https://discuss.elastic.co/t/problems-with-configure-multi-pipelines-logtash-v6-1-2/121146/1 "2018-02-22T23:56:42Z")

</div>

Hi, I trying configure logstash with multipipelines and cannot makes works, i follow the documentations and is doesnt work... I comment the path.config in the logstash.yml and add the path's of the each pipeline in pipelines.yml..

> logstash.yml  
> #path.config: /usr/share/logstash/pipeline  
> path.data: /usr/share/logstash/  
> xpack.monitoring.elasticsearch.url: [http://x.x.x.x:9200](http://x.x.x.x:9200)  
> xpack.monitoring.elasticsearch.username: xxxxxxxxxxx  
> xpack.monitoring.elasticsearch.password: xxxxxxxxxxx

> pipelines.yml

- pipeline.id: microservicios  
path.config: "/usr/share/logstash/pipeline/microservicios.conf"  
pipeline.workers: 1
- pipeline.id: beats  
path.config: "/usr/share/logstash/pipeline/beats.conf"  
pipeline.workers: 1
- pipeline.id: test  
pipeline.config: "/usr/share/logstash/pipeline/logstash.conf"

when i try up logstash service crash

> Sending Logstash's logs to /usr/share/logstash/logs which is now configured via log4j2.properties  
> [2018-02-22T23:55:22,468][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/usr/share/logstash/modules/fb\_apache/configuration"}  
> [2018-02-22T23:55:22,498][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"/usr/share/logstash/modules/netflow/configuration"}  
> [2018-02-22T23:55:23,989][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"arcsight", :directory=\>"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/x-pack-6.1.2-java/modules/arcsight/configuration"}  
> [2018-02-22T23:55:27,435][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.1.2"}  
> [ERROR] 2018-02-22 23:55:27.681 [Ruby-0-Thread-1: /usr/share/logstash/lib/bootstrap/environment.rb:6] sourceloader - Could not fetch all the sources {:exception=\>ArgumentError, :message=\>"Setting "pipeline.config" hasn't been registered", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:32:in `get_setting'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:65:in `set\_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:84:in `block in merge'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:84:in `merge'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:19:in `block in pipeline\_configs'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:18:in `pipeline\_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source\_loader.rb:59:in `block in fetch'", "org/jruby/RubyArray.java:2481:in `collect'", "/usr/share/logstash/logstash-core/lib/logstash/config/source\_loader.rb:58:in `fetch'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:148:in `converge\_state\_and\_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:343:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}  
> [2018-02-22T23:55:27,706][ERROR][logstash.agent] An exception happened when converging configuration {:exception=\>RuntimeError, :message=\>"Could not fetch the configuration, message: Setting "pipeline.config" hasn't been registered", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/agent.rb:155:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:343:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

Help Me please..

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [February 23, 2018, 4:10am UTC](https://discuss.elastic.co/t/problems-with-configure-multi-pipelines-logtash-v6-1-2/121146/2 "2018-02-23T04:10:07Z")

</div>

please remove quotes("") and try.

path.config: "/usr/share/logstash/pipeline/microservicios.conf"

path.config:/usr/share/logstash/pipeline/microservicios.conf ------try this

---

<div class="post-metadata">

**Author:** ![agonex](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@agonex](https://discuss.elastic.co/u/agonex)\
**Post date:** [February 23, 2018, 3:22pm UTC](https://discuss.elastic.co/t/problems-with-configure-multi-pipelines-logtash-v6-1-2/121146/3 "2018-02-23T15:22:48Z")

</div>

Thank you, I Remove (") and not work.

> Sending Logstash's logs to /usr/share/logstash/logs which is now configured via log4j2.properties  
> [2018-02-23T15:21:08,351][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/usr/share/logstash/modules/fb\_apache/configuration"}  
> [2018-02-23T15:21:08,385][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"/usr/share/logstash/modules/netflow/configuration"}  
> [2018-02-23T15:21:10,143][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"arcsight", :directory=\>"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/x-pack-6.1.2-java/modules/arcsight/configuration"}  
> [2018-02-23T15:21:13,766][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.1.2"}  
> [ERROR] 2018-02-23 15:21:14.013 [Ruby-0-Thread-1: /usr/share/logstash/lib/bootstrap/environment.rb:6] sourceloader - Could not fetch all the sources {:exception=\>ArgumentError, :message=\>"Setting "pipeline.config" hasn't been registered", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:32:in `get_setting'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:65:in `set\_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:84:in `block in merge'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:84:in `merge'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:19:in `block in pipeline\_configs'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:18:in `pipeline\_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source\_loader.rb:59:in `block in fetch'", "org/jruby/RubyArray.java:2481:in `collect'", "/usr/share/logstash/logstash-core/lib/logstash/config/source\_loader.rb:58:in `fetch'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:148:in `converge\_state\_and\_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:343:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}  
> [2018-02-23T15:21:14,043][ERROR][logstash.agent] An exception happened when converging configuration {:exception=\>RuntimeError, :message=\>"Could not fetch the configuration, message: Setting "pipeline.config" hasn't been registered", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/agent.rb:155:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:343:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 23, 2018, 3:54pm UTC](https://discuss.elastic.co/t/problems-with-configure-multi-pipelines-logtash-v6-1-2/121146/4 "2018-02-23T15:54:41Z")

</div>

> [@agonex](#):
>
> "Setting "pipeline.config" hasn't been registered"

For the test pipeline you have pipeline.config where you should have path.config.

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [February 26, 2018, 3:31am UTC](https://discuss.elastic.co/t/problems-with-configure-multi-pipelines-logtash-v6-1-2/121146/5 "2018-02-26T03:31:38Z")

</div>

paste ur config file here???

---

<div class="post-metadata">

**Author:** ![agonex](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@agonex](https://discuss.elastic.co/u/agonex)\
**Post date:** [February 26, 2018, 6:03pm UTC](https://discuss.elastic.co/t/problems-with-configure-multi-pipelines-logtash-v6-1-2/121146/6 "2018-02-26T18:03:26Z")

</div>

Logstash.yml

> http.host: "0.0.0.0"  
> path.config: /usr/share/logstash/pipeline  
> #path.data: /usr/share/logstash/  
> xpack.monitoring.elasticsearch.url: [http://x.x.x.x:9200](http://x.x.x.x:9200)  
> xpack.monitoring.elasticsearch.username: xxxxxxx  
> xpack.monitoring.elasticsearch.password: xxxxxxxx

Logstash.conf

> input{  
> tcp {  
> type =\> "test"  
> port =\> 9900  
> codec =\> "json"  
> }  
> }  
> output {  
> if [type] == "test"{  
> elasticsearch {  
> hosts =\> ["x.x.x.x:9200"]  
> user =\> xxxxxxx  
> password =\> xxxxxx  
> index =\> "prueba"  
> }  
> }  
> stdout {  
> codec =\> rubydebug  
> }  
> }

Beats.conf

> input {  
> beats {  
> port =\> 9400  
> type =\> "beats"  
> }  
> }  
> output {  
> if [type] == "beats"{  
> elasticsearch {  
> hosts =\> ["x.x.x.x:9200"]  
> user =\> xxxxx  
> password =\> xxxxxx  
> index =\> "beats-%{+YYYY.MM.dd}"  
> }  
> }  
> else {  
> elasticsearch {  
> hosts =\> ["x.x.x.x:9200"]  
> user =\> xxxxx  
> password =\> xxxxx  
> index =\> "was-%{+YYYY.MM.dd}"  
> }  
> }  
> stdout {  
> codec =\> rubydebug  
> }  
> }

microservicios.yml

> input{  
> tcp {  
> type =\> "microservices"  
> port =\> 9500  
> codec =\> multiline{  
> pattern =\> "^(?%{MONTHDAY}[-]%{MONTHNUM}[-]%{YEAR})|"  
> negate =\> true  
> what =\> "previous"  
> }  
> }  
> tcp {  
> type =\> "was"  
> port =\> 9700  
> codec =\> multiline{  
> pattern =\> "^(?%{MONTHDAY}[-]%{MONTHNUM}[-]%{YEAR})"  
> negate =\> true  
> what =\> "previous"  
> }  
> }  
> tcp {  
> type =\> "monitoring"  
> port =\> 18080  
> }  
> tcp {  
> type =\> "atla\_services"  
> port =\> 9800  
> codec =\> "json"  
> }  
> }  
> filter {  
> if [type] == "microservices"{  
> grok {  
> match =\> { "message" =\> "(?%{MONTHDAY}[-]%{MONTHNUM}[-]%{YEAR})?|(?%{HOUR}:%{MINUTE}:%{SECOND})?|(%{NONNEGINT:sss})?|(%{GREEDYDATA:cic})?|(%{GREEDYDATA:idc})?|(%{UUID:sesUID})?|(%{UUID:trnUID})?|(%{NONNEGINT:opnNro})?|(%{NONNEGINT:opnNroHost})?|(%{IP:servIp})?|(%{HOSTNAME:servNom})?|(%{GREEDYDATA:class})?|(%{NONNEGINT:idClass})?|(%{LOGLEVEL:loglevel})?|(%{GREEDYDATA:namespace}) ?|(%{WORD:method})?|(%{GREEDYDATA:msgMicroservices})?"}  
> }  
> }  
> if [type] == "was"{  
> grok {  
> match =\> { "message" =\> "%{DATE:fecha\_was}? %{TIME:hora\_was}? [(%{GREEDYDATA:cic})?] [(%{GREEDYDATA:session})?] : [%{LOGLEVEL:loglevel}] (%{GREEDYDATA:namespace}) ? - %{GREEDYDATA:msgWas}?" }  
> }  
> }  
> date {  
> match =\> ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]  
> }  
> }  
> output {  
> if [type] == "atla\_services"{  
> elasticsearch {  
> hosts =\> ["x.x.x.x.x:9200"]  
> user =\> xxxxxx  
> password =\> xxxxxx  
> index =\> "atlas-%{+YYYY.MM.dd}"  
> }  
> }  
> else {  
> elasticsearch {  
> hosts =\> ["x.x.x.x:9200"]  
> user =\> xxxxxxx  
> password =\> xxxxxxxx  
> }  
> stdout {  
> codec =\> rubydebug  
> }  
> }  
> }

pipelines.yml

```
     - pipeline.id: microservicios
       path.config: /usr/share/logstash/pipeline/microservicios.conf
       pipeline.workers: 1
     - pipeline.id: beats
       path.config: /usr/share/logstash/pipeline/beats.conf
       pipeline.workers: 1
     - pipeline.id: test
       pipeline.config: /usr/share/logstash/pipeline/logstash.conf

```

Thats all my config

---

<div class="post-metadata">

**Author:** ![agonex](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@agonex](https://discuss.elastic.co/u/agonex)\
**Post date:** [February 26, 2018, 8:55pm UTC](https://discuss.elastic.co/t/problems-with-configure-multi-pipelines-logtash-v6-1-2/121146/7 "2018-02-26T20:55:57Z")

</div>

I read some internet forums and somebody suggets comment the path.config in losgtash.yml to force logstash read the content in pipelines.yml.. i show the logs... with that line commented in logs not appear ingnore pipelines.yml

> [2018-02-26T20:49:15,934][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.1.2"}  
> [ERROR] 2018-02-26 20:49:16.183 [Ruby-0-Thread-1: /usr/share/logstash/lib/bootstrap/environment.rb:6] sourceloader - Could not fetch all the sources {:exception=\>ArgumentError, :message=\>"Setting "pipeline.config" hasn't been registered", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:32:in `get_setting'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:65:in `set\_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:84:in `block in merge'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:84:in `merge'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:19:in `block in pipeline\_configs'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:18:in `pipeline\_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source\_loader.rb:59:in `block in fetch'", "org/jruby/RubyArray.java:2481:in `collect'", "/usr/share/logstash/logstash-core/lib/logstash/config/source\_loader.rb:58:in `fetch'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:148:in `converge\_state\_and\_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:343:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}  
> [2018-02-26T20:49:16,211][ERROR][logstash.agent] An exception happened when converging configuration {:exception=\>RuntimeError, :message=\>"Could not fetch the configuration, message: Setting "pipeline.config" hasn't been registered", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/agent.rb:155:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:343:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

thanks

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [February 27, 2018, 3:23am UTC](https://discuss.elastic.co/t/problems-with-configure-multi-pipelines-logtash-v6-1-2/121146/8 "2018-02-27T03:23:00Z")

</div>

> [@agonex](#):
>
> http.host: "0.0.0.0"
> 
> path.config: /usr/share/logstash/pipeline
> 
> #path.data: /usr/share/logstash/
> 
> xpack.monitoring.elasticsearch.url: [http://x.x.x.x:9200](http://x.x.x.x:9200)
> 
> xpack.monitoring.elasticsearch.username: xxxxxxx
> 
> xpack.monitoring.elasticsearch.password: xxxxxxxx

http.host: "0.0.0.0"  
\<path.config: /usr/share/logstash/pipeline ----u shud give the path of your config file here like ur config file is Logstash.conf then provide the path of this file.\>

---

<div class="post-metadata">

**Author:** ![agonex](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@agonex](https://discuss.elastic.co/u/agonex)\
**Post date:** [February 27, 2018, 2:28pm UTC](https://discuss.elastic.co/t/problems-with-configure-multi-pipelines-logtash-v6-1-2/121146/9 "2018-02-27T14:28:28Z")

</div>

Ok, thats it the mistake... I sorry i can't believe it... i don't see the config with attention

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 27, 2018, 2:28pm UTC](https://discuss.elastic.co/t/problems-with-configure-multi-pipelines-logtash-v6-1-2/121146/10 "2018-03-27T14:28:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.