Logtash |7.12.1 | Multiple pipelines

Elastic Stack Logstash
1

Hello All
I have to configure multiple pipelines for which I carried out the changes in pipelines.yml with simple test configuration, after removing all the code to debug to make it work. Below is the sample configuration now, initially, I specified two pipelines when it did not work, I reduced it to one. However, I am facing the below error everytime. Any inputs will be very helpful. Thanks

pipelines.yml

image
image849×59 3.07 KB

test.conf
image

Error
C:\tools\logstash-7.12.1\bin>logstash
Using JAVA_HOME defined java: C:\Program Files\Java\jdk-11.0.11
WARNING, using JAVA_HOME while Logstash distribution comes with a bundled JDK
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to C:/tools/logstash-7.12.1/logs which is now configured via log4j2.properties
[2021-05-21T03:52:18,708][INFO ][logstash.runner ] Log4j configuration path used is: C:\tools\logstash-7.12.1\config\log4j2.properties
[2021-05-21T03:52:18,735][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.12.1", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc Java HotSpot(TM) 64-Bit Server VM 11.0.11+9-LTS-194 on 11.0.11+9-LTS-194 +indy +jit [mswin32-x86_64]"}
[2021-05-21T03:52:21,190][ERROR][logstash.config.sourceloader] Could not fetch all the sources {:exception=>NoMethodError, :message=>"undefined method each' for #<String:0x20ddda4a>", :backtrace=>["C:/tools/logstash-7.12.1/logstash-core/lib/logstash/settings.rb:145:in merge_pipeline_settings'", "C:/tools/logstash-7.12.1/logstash-core/lib/logstash/config/source/multi_local.rb:36:in block in pipeline_configs'", "org/jruby/RubyArray.java:2577:in map'", "C:/tools/logstash-7.12.1/logstash-core/lib/logstash/config/source/multi_local.rb:34:in pipeline_configs'", "C:/tools/logstash-7.12.1/logstash-core/lib/logstash/config/source_loader.rb:76:in block in fetch'", "org/jruby/RubyArray.java:2572:in collect'", "C:/tools/logstash-7.12.1/logstash-core/lib/logstash/config/source_loader.rb:75:in fetch'", "C:/tools/logstash-7.12.1/logstash-core/lib/logstash/agent.rb:188:in converge_state_and_update'", "C:/tools/logstash-7.12.1/logstash-core/lib/logstash/agent.rb:126:in execute'", "C:/tools/logstash-7.12.1/logstash-core/lib/logstash/runner.rb:409:in block in execute'", "C:/tools/logstash-7.12.1/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/task.rb:24:in block in initialize'"]}
[2021-05-21T03:52:21,283][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2021-05-21T03:52:21,454][ERROR][logstash.agent ] An exception happened when converging configuration {:exception=>RuntimeError, :message=>"Could not fetch the configuration, message: undefined method `each' for #String:0x20ddda4a"}
[2021-05-21T03:52:21,917][INFO ][logstash.runner ] Logstash shut down.
[2021-05-21T03:52:21,949][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.13.0.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.13.0.jar:?]
at C_3a_.tools.logstash_minus_7_dot_12_dot_1.lib.bootstrap.environment.(C:\tools\logstash-7.12.1\lib\bootstrap\environment.rb:89) ~[?:?]

2

Welcome!

I moved your question to #elastic-stack:logstash.

Please don't post images of text as they are hard to read, may not display correctly for everyone, and are not searchable.

Instead please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

This is the icon to use if you are not using markdown format:

image
image764×92 16.4 KB

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
Please update your post.

3

Any update on this would be appreciated.

4

Do not use the stdin input plugin as I do not think this will work. Pick another one and see if that makes a difference.

5

YAML uses a colon and a space to separate keys and values. That error message is telling you that you have

- pipeline.id:someid
  path.config:"/some/path"

instead of

- pipeline.id: someid
  path.config:  "/some/path"

If you include the space in one line but not the other you get the equally uninformative error

ERROR: Failed to read pipelines yaml file. Location: /etc/logstash/pipelines.yml
2 Likes
6

Thanks @Badger , this is exactly an issue was. How I did not see at the fist glance, a silly mistage. Thank you!! for your response.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.