Hi,
I am testing a pipelined configuration using the output isolator pattern. While this I am still using pipeline batch/worker settings from logstash.yml and would like to use pipeline.ordered for one of the outputs.
When I tried this, and dropped pipeline.workers to 1 for that output logstash failed to start with:
[ERROR][logstash.config.sourceloader] Could not fetch all the sources {:exception=>ArgumentError, :message=>"Setting \"pipeline.ordered\" hasn't been registered", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:36:in `get_setting'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:69:in `set_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:88:in `block in merge'", "org/jruby/RubyHash.java:1419:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:88:in `merge'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:18:in `block in pipeline_configs'", "org/jruby/RubyArray.java:2577:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:17:in `pipeline_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:61:in `block in fetch'", "org/jruby/RubyArray.java:2572:in `collect'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:60:in `fetch'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:148:in `converge_state_and_update'", "/usr/share/logstash/x-pack/lib/monitoring/internal_pipeline_source.rb:49:in `enable_monitoring'", "/usr/share/logstash/x-pack/lib/monitoring/internal_pipeline_source.rb:39:in `update_license_state'", "/usr/share/logstash/x-pack/lib/license_checker/licensed.rb:39:in `setup_license_checker'", "/usr/share/logstash/x-pack/lib/monitoring/internal_pipeline_source.rb:23:in `initialize'", "/usr/share/logstash/x-pack/lib/monitoring/monitoring.rb:98:in `after_agent'", "org/logstash/execution/EventDispatcherExt.java:71:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:360:in `execute'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:242:in `run'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "/usr/share/logstash/lib/bootstrap/environment.rb:73:in `<main>'"]}
Settings applied to the output in question:
- pipeline.id: output2
queue.type: persisted
pipeline.workers: 1
pipeline.batch.size: 8000
pipeline.ordered: true
......
Note: Global workers/batch settings are still set in logstash.yml. Logstash version is 7.2.0
Am I misinterpreting how worker settings are applied in a multi-pipline configuration?
Thx
D