Hello guys i'm a little bit stuck with my config logstash
i try to have multiple pipeline one for AD events and Palo alto syslog event
so i've configure the file pipeline.yml like this
i've a question do i need to set up the path of pipeline.yml in logstash.yml or he read this file without any other configuration ?
- pipeline.id: Filter_AD
path.config: /etc/logstash/conf.d/filter_AD.conf
queue.type: persisted
path.queue: /var/log/logstash/logstash_queue/filter_ad
config.reload.automatic: true
pipeline.batch.size: 125
pipeline.batch.delay: 50
pipeline.workers: 1
- pipeline.id: filter_PAN
path.config: /etc/logstash/conf.d/PAN-OS.conf
queue.type: persisted
path_queue: /var/log/logstash/logstash_queue/PAN-OS
config.reload.automatic: true
pipeline.batch.size: 125
pipeline.batch.delay: 50
pipeline.workers: 1
and when i restart logstash service i've this error in log file
[2019-03-13T10:24:53,380][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2019-03-13T10:24:53,382][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2019-03-13T10:24:58,385][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2019-03-13T10:24:58,387][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2019-03-13T10:24:58,475][DEBUG][logstash.config.source.multilocal] Reading pipeline configurations from YAML {:location=>"/etc/logstash/pipelines.yml"}
[2019-03-13T10:24:58,479][ERROR][logstash.config.sourceloader] Could not fetch all the sources {:exception=>ArgumentError, :message=>"Setting \"path_queue\" hasn't been registered", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:36:in `get_setting'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:69:in `set_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:88:in `block in merge'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:88:in `merge'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:18:in `block in pipeline_configs'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:17:in `pipeline_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:61:in `block in fetch'", "org/jruby/RubyArray.java:2481:in `collect'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:60:in `fetch'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:150:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:118:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/interval.rb:18:in `interval'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:107:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:362:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}
[2019-03-13T10:24:58,479][DEBUG][logstash.agent ] Could not fetch the configuration to converge, will retry {:message=>"Setting \"path_queue\" hasn't been registered", :retrying_in=>10.0}
and when i comment the second pipeline with # i can received event form winlogbeat
thx for your help