Multiple pipelines on Logstash 6.6


(Rafael Montesinos) #1

Regards
I have a problem when starting an instance of Logstash 6.6 with multiple pipelines, each pipeline will query an Oracle DB, these queries will generate a set of JSON data that will be showed at Kibana in a Dashboard. Each of these data pipelines separately works fine, but I need to converge all information. For this I want to take advantage of the utility of multiple pipelines in a single instance of Logstash that since version 6.x exists.

The logstash.yml contains:
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.url: "https://localhost:9200"
xpack.monitoring.elasticsearch.sniffing: true
xpack.monitoring.collection.interval: 10s
xpack.monitoring.collection.pipeline.details.enabled: true

pipelines.yml contains:
- pipeline.id:PROSEM_DEMANDA_5_ANNO
pipeline.workers: 1
pipeline.output.workers: 1
pipeline.batch.size: 1
queue.type: memory
path.config: "config/pipelines/v_prosem_CAB_demanda_anno.cfg"
- pipeline.id:PROSEM_CONTRATACIONES_5_ANNO
pipeline.workers: 2
pipeline.output.workers: 2
pipeline.batch.size: 1
queue.type: memory
path.config: "config/pipelines/v_prosem_cab_contr.cfg"

the log spits the following error:

[2019-03-15T12:36:57,854][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
[2019-03-15T12:37:16,991][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>#<ArgumentError: Setting "" hasn't been registered>, :backtrace=>["C:/elk/logstash/logstash-core/lib/logstash/settings.rb:36:in `get_setting'", "C:/elk/logstash/logstash-core/lib/logstash/settings.rb:69:in `set_value'", "C:/elk/logstash/logstash-core/lib/logstash/settings.rb:88:in `block in merge'", "org/jruby/RubyHash.java:1343:in `each'", "C:/elk/logstash/logstash-core/lib/logstash/settings.rb:88:in `merge'", "C:/elk/logstash/logstash-core/lib/logstash/settings.rb:137:in `validate_all'", "C:/elk/logstash/logstash-core/lib/logstash/runner.rb:278:in `execute'", "C:/elk/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "C:/elk/logstash/logstash-core/lib/logstash/runner.rb:237:in `run'", "C:/elk/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "C:\\elk\\logstash\\lib\\bootstrap\\environment.rb:73:in `<main>'"]}
[2019-03-15T12:37:17,006][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
[2019-03-15T12:37:36,625][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>#<ArgumentError: Setting "" hasn't been registered>, :backtrace=>["C:/elk/logstash/logstash-core/lib/logstash/settings.rb:36:in `get_setting'", "C:/elk/logstash/logstash-core/lib/logstash/settings.rb:69:in `set_value'", "C:/elk/logstash/logstash-core/lib/logstash/settings.rb:88:in `block in merge'", "org/jruby/RubyHash.java:1343:in `each'", "C:/elk/logstash/logstash-core/lib/logstash/settings.rb:88:in `merge'", "C:/elk/logstash/logstash-core/lib/logstash/settings.rb:137:in `validate_all'", "C:/elk/logstash/logstash-core/lib/logstash/runner.rb:278:in `execute'", "C:/elk/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "C:/elk/logstash/logstash-core/lib/logstash/runner.rb:237:in `run'", "C:/elk/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "C:\\elk\\logstash\\lib\\bootstrap\\environment.rb:73:in `<main>'"]}
[2019-03-15T12:37:36,625][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

I have checked the way to tell the attribute path.config: "config/pipelines/v_prosem_cab_contr.cfg" on Windows :tired_face: systems as I see at : Logstash Multiple Pipelines Doesn't work

All help is welcome thank you in advance