Still getting same issues.
Tired swapping out the pipelines.yml file whilst logstash was still running and its reporting:
[2020-01-22T09:33:11,758][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2020-01-22T09:33:11,759][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2020-01-22T09:33:12,816][DEBUG][logstash.config.source.multilocal] Reading pipeline configurations from YAML {:location=>"/etc/logstash/pipelines.yml"}
[2020-01-22T09:33:12,819][ERROR][logstash.config.sourceloader] Could not fetch all the sources {:exception=>LogStash::ConfigurationError, :message=>"Failed to read pipelines yaml file. Location: /etc/logstash/pipelines.yml, Exception: #<Psych::SyntaxError: (<unknown>): could not find expected ':' while scanning a simple key at line 29 column 1>", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:82:in `read_pipelines_from_yaml'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:67:in `retrieve_yaml_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:17:in `pipeline_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:61:in `block in fetch'", "org/jruby/RubyArray.java:2579:in `collect'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:60:in `fetch'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:149:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:114:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:18:in `interval'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:103:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:369:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}
[2020-01-22T09:33:12,822][DEBUG][logstash.agent ] Could not fetch the configuration to converge, will retry {:message=>"Failed to read pipelines yaml file. Location: /etc/logstash/pipelines.yml,
Seems its getting stuck on this
(<unknown>): could not find expected ':'
Yet
- pipeline.id: 1st-index
is perfectly fine.. if i alter the variables and comment out stuff and make it work reading a flat file it parses... so the pipeline object here is workable... which suggests to me that its something previous to this
output { pipeline { send_to => ["main-index", "1st-index"] } }
IS it maybe that i can't fork this correctly or it parse to main-index, but can't pipe to 1st-index? ( i've altered the pipeline names as well removing the - just in case and that doesn't work