Logstash unable to process pipelines.yml

So i'm wrestling with a new problem today... Logstash 7.5.0

recently upgraded my test ELK stack from 7.4.2 to 7.5.0

Immediately hit an issue ingestion data with my setup.

the error i get is

Dec 03 15:14:49 ELK01 logstash[20424]: [2019-12-03T15:14:49,217][DEBUG][logstash.agent           ] Converging pipelines state {:actions_count=>1}
Dec 03 15:14:49 ELK01 logstash[20424]: [2019-12-03T15:14:49,224][DEBUG][logstash.agent           ] Executing action {:action=>LogStash::PipelineAction::Create/pipeline_id:main}
Dec 03 15:14:49 ELK01 logstash[20424]: [2019-12-03T15:14:49,285][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
Dec 03 15:14:49 ELK01 logstash[20424]: [2019-12-03T15:14:49,290][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
Dec 03 15:14:49 ELK01 logstash[20424]: [2019-12-03T15:14:49,308][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :**message=>"Expected one of [ \\t\\r\\n], \"#\", \"input\", \"filter\", \"output\" at line 1, column 1 (byte 322) after #- pipeline.id: main\n#**  path.config: \"/etc/logstash/conf.d/*.conf\"\n#  pipeline.workers: 1\n", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:41:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:49:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `block in compile_sources'", "org/jruby/RubyArray.java:2584:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:10:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:156:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:26:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:326:in `block in converge_state'"]}
Dec 03 15:14:54 ELK01 logstash[20424]: [2019-12-03T15:14:54,317][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
Dec 03 15:14:54 ELK01 logstash[20424]: [2019-12-03T15:14:54,324][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}

My pipelines.yml is thus:

- pipeline.id: dataingest
  path.config: "/etc/logstash/conf.d/*.conf"
  pipeline.workers: 1

And my logstash.yml is thus:

pipeline.batch.size: 125
pipeline.batch.delay: 5
node.name: logstash
path.data: /var/lib/logstash
pipeline.unsafe_shutdown: true
path.config: /etc/logstash/pipelines.yml
config.reload.automatic: true
config.reload.interval: 10s
config.debug: true
http.host: "127.0.0.1"
http.port: 9600-9700
log.level: debug
path.logs: /var/log/logstash
pipeline.separate_logs: true

I've checked with the 7.5.0 guides and there's nothing there comparing to my configs thats amiss... yet the error message=>"Expected one of [ \t\r\n], "#", "input", "filter", "output" at line 1, column 1 (byte 322) after #- pipeline.id: main\n# is basically complaining that there's no config.string: "input { } filter { } output { }" like entry.... but we have these in our logstash configs already... has there been a massive change in how the process works that hasn't yet made it into the logstash guides yet?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.