Logstash terminating pipelines error "const_missing, block in JDBC"

SamehSaeed · December 20, 2023, 12:17pm

Hello, I have a problem when running logstash with multiple pipelines (around 70). Logstash will always terminate some of them if i run more than 30 concurrently

1- First error :

[ERROR][logstash.javapipeline    ][bkgetivrnumbersoffers] Pipeline error {:pipeline_id=>"bkgetivrnumbersoffers", :exception=>#<NameError: uninitialized constant Sequel::JDBC::Oracle::DatabaseMethods
Did you mean?  Sequel::JDBC::Database>, :backtrace=>["org/jruby/RubyModule.java:4309:in `const_missing'" "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc/oracle.rb:11:in `block in JDBC'",

2 - Second error after adjusting pipeline.workers to 5 :

[2023-12-20T11:13:16,616][ERROR][logstash.outputs.elasticsearch] Invalid setting for elasticsearch output plugin:

  output {
    elasticsearch {
      # This setting must be a ["inline", "indexed", "file"]
      # Expected one of ["inline", "indexed", "file"], got [nil]
      script_type => [nil]
      ...
    }

}

i tried adjusting pipeline workers to 30, but still i cant run more than 35 pipelines concurrently.
After switching driver to ojdbc8, the number of concurrent went up to 63(this number varies after each restart) and logstash terminated the rest, but still that's about quarter of what i'm about to run in the future.
logstash sample :

input {
    jdbc {
	 clean_run => true
        jdbc_driver_library => "D:\ELK 8.10.2\logstash-conf\ojdbc8.jar"
        jdbc_driver_class => "oracle.jdbc.driver.OracleDriver"
        jdbc_connection_string => "*"
        jdbc_user => "*"
		jdbc_password => "*"
		jdbc_validate_connection => true
		jdbc_default_timezone => "Asia/Riyadh"
		jdbc_pool_timeout => 120
		jdbc_validation_timeout => 120
        schedule => "*/5 * * * *"
		connection_retry_attempts => 3
		last_run_metadata_path => "D:/ELK 8.10.2/logstash-8.10.2/config/metadata/bk_services/bkvalidateotpcancellation-lastrun" 
        statement => "
SELECT * FROM (
SELECT 
'Validate-OTP-Cancellation' AS NODE,
'PublicPostOtpVerification' AS SERVICE_NAME,
CALL_STRAT_DATE_TIME AS CALL_START_TIME,
 (CAST(CALL_STRAT_DATE_TIME AS DATE) - DATE '1970-01-01')*24*60*60*1000 + MOD( EXTRACT( SECOND FROM CALL_STRAT_DATE_TIME ), 1 )  AS UNIX_EPOCH_TIME,
SESSION_ID,
CHANNEL,
CASE WHEN (INSTR(CALL_HISTORY, 'Check GSM Customer Service/prepaid', -1) > 0 OR INSTR(CALL_HISTORY, 'Check GSM Customer Type/prepaid', -1) > 0 OR INSTR(CALL_HISTORY, 'Check Disconnect Customer Type/prepaid', -1) > 0) THEN 'PREPAID' WHEN INSTR(CALL_HISTORY, 'Check GSM Customer Type For IE Customers/prepaid', -1) > 0 THEN 'IE PREPAID' WHEN (INSTR(CALL_HISTORY, 'Check GSM Customer Service/postpaid', -1) > 0 OR INSTR(CALL_HISTORY, 'Check GSM Customer Type/postpaid', -1) > 0 OR INSTR(CALL_HISTORY, 'Check Disconnect Customer Type/postpaid', -1) > 0) THEN 'POSTPAID' WHEN INSTR(CALL_HISTORY, 'Check GSM Customer Type For IE Customers/postpaid', -1) > 0 THEN 'IE POSTPAID' WHEN (INSTR(CALL_HISTORY, 'Check Phone Type/landline', -1) > 0 OR INSTR(CALL_HISTORY, 'Check Disconnect Customer Type/landline', -1) > 0) THEN 'LANDLINE' WHEN INSTR(CALL_HISTORY, 'Check Phone Type For IE Customers/landline', -1) > 0 THEN 'IE LANDLINE' END AS PHONE_TYPE,
TO_CHAR(SUBSTR(
                CALL_HISTORY,
                INSTR(CALL_HISTORY, 'Validate OTP Cancellation/') + LENGTH('Validate OTP Cancellation/'),
                INSTR(CALL_HISTORY, ';', INSTR(CALL_HISTORY, 'Validate OTP Cancellation/')) - (INSTR(CALL_HISTORY, 'Validate OTP Cancellation/') + LENGTH('Validate OTP Cancellation/'))
              )) AS STATUS,
              REGEXP_COUNT(CALL_HISTORY, 'Validate OTP Cancellation/') AS NUMBER_OF_HITS
FROM CALL_HISTORY_DETAILS_VIEW 
WHERE INSTR(CALL_HISTORY, 'Validate OTP Cancellation/') > 0 
AND CALL_STRAT_DATE_TIME BETWEEN SYSDATE - INTERVAL '120' MINUTE AND SYSDATE - INTERVAL '60' MINUTE 
)
WHERE UNIX_EPOCH_TIME > :sql_last_value
			"
        use_column_value => true
		tracking_column => "unix_epoch_time"
		tracking_column_type => "numeric"
		tags => ["bkvalidateotpcancellation"]
    }
}
filter
{
	date
	{
		match => ["@timestamp", "ISO8601"]
		timezone => "Asia/Riyadh"
		target => "@timestamp_adjusted"
	}
}

output{
 
    elasticsearch {	
        hosts => ["http://localhost:9200/"]
		index => "bkvalidateotpcancellation-%{+YYYY.MM.dd}"
		document_id => "%{unix_epoch_time}"
		user => "*"
		password => "*"
		ssl => false
		ssl_certificate_verification => false
    }
   
}

Note : All pipelines have exactly the same configuration, only changing the substring inside the select statement.

So my questions :
1 - Is this the correct approach? or do i need to use one pipeline for all these statements
2 - How can i run more pipelines at the same time?
3 - if logstash terminates a pipeline for some reason, how do i start it again without restarting logstash? and how do i even detect that a pipeline was terminated overnight.

ELK stack version : 8.10.2
Using on windows, as a service (through NSSM)

Badger · December 20, 2023, 7:25pm

You say you got to 63 pipelines and that is a quarter of what you need. If you run 250 pipelines with 30 worker threads each that is 7500 threads. The memory demand for thread stacks will be monstrous, and the context switching will thrash the on-chip memory caches (L3 etc.). Even if it worked the performance would be terrible.

I would suggest setting pipeline.workers to 1. Then tell us what logstash logs.

For the script_type => [nil] error are you asserting that you have not set script_type on the output configuration and so the default value of inline should be used?

SamehSaeed · December 21, 2023, 7:17am

Hello Badger, thank you for your reply.

Yes, i did not set "script_type" and i do not know what it does so im gonna check the documentation.

Setting pipeline workers to (1) seems to be working really well and RAM consumption is not spiking also. But still 2 pipelines were terminated

[2023-12-21T09:18:28,365][INFO ][logstash.javapipeline    ][bksubmitmcamp] Pipeline Java execution initialization time {"seconds"=>2.82}
[2023-12-21T09:18:30,170][ERROR][logstash.javapipeline    ][bksubmitrefreshallservices] Pipeline error {:pipeline_id=>"bksubmitrefreshallservices", :exception=>#<NameError: uninitialized constant Sequel::JDBC::Oracle::DatabaseMethods
Did you mean?  Sequel::JDBC::Database>, :backtrace=>["org/jruby/RubyModule.java:4309:in `const_missing'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc/oracle.rb:11:in `block in JDBC'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:379:in `adapter_initialize'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/misc.rb:169:in `initialize'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/connecting.rb:57:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/core.rb:124:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:123:in `block in jdbc_connect'", "org/jruby/RubyKernel.java:1586:in `loop'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:120:in `jdbc_connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:163:in `open_jdbc_connection'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/inputs/jdbc.rb:309:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-mixin-ecs_compatibility_support-1.3.0-java/lib/logstash/plugin_mixins/ecs_compatibility_support/target_check.rb:48:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:237:in `block in register_plugins'", "org/jruby/RubyArray.java:1987:in `each'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:236:in `register_plugins'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:395:in `start_inputs'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:320:in `start_workers'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:194:in `run'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:146:in `block in start'"], "pipeline.sources"=>["D:/ELK 8.10.2/logstash-8.10.2/config/conf.d/bk_services/bksubmitrefreshallservices_config.conf"], :thread=>"#<Thread:0x175fc348 D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2023-12-21T09:18:30,206][ERROR][logstash.javapipeline    ][bkcheckreconnecttries] Pipeline error {:pipeline_id=>"bkcheckreconnecttries", :exception=>#<NameError: uninitialized constant Sequel::JDBC::Oracle::DatabaseMethods
Did you mean?  Sequel::JDBC::Database>, :backtrace=>["org/jruby/RubyModule.java:4309:in `const_missing'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc/oracle.rb:11:in `block in JDBC'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:379:in `adapter_initialize'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/misc.rb:169:in `initialize'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/connecting.rb:57:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/core.rb:124:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:123:in `block in jdbc_connect'", "org/jruby/RubyKernel.java:1586:in `loop'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:120:in `jdbc_connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:163:in `open_jdbc_connection'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/inputs/jdbc.rb:309:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-mixin-ecs_compatibility_support-1.3.0-java/lib/logstash/plugin_mixins/ecs_compatibility_support/target_check.rb:48:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:237:in `block in register_plugins'", "org/jruby/RubyArray.java:1987:in `each'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:236:in `register_plugins'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:395:in `start_inputs'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:320:in `start_workers'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:194:in `run'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:146:in `block in start'"], "pipeline.sources"=>["D:/ELK 8.10.2/logstash-8.10.2/config/conf.d/bk_services/bkcheckreconnecttries_config.conf"], :thread=>"#<Thread:0x23313f3 D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2023-12-21T09:18:30,396][INFO ][logstash.javapipeline    ][bksubmitrefreshallservices] Pipeline terminated {"pipeline.id"=>"bksubmitrefreshallservices"}
[2023-12-21T09:18:30,396][INFO ][logstash.javapipeline    ][bkcheckreconnecttries] Pipeline terminated {"pipeline.id"=>"bkcheckreconnecttries"}
[2023-12-21T09:18:30,475][ERROR][logstash.agent           ] Failed to execute action {:id=>:bkcheckreconnecttries, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<bkcheckreconnecttries>, action_result: false", :backtrace=>nil}
[2023-12-21T09:18:30,475][ERROR][logstash.agent           ] Failed to execute action {:id=>:bksubmitrefreshallservices, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<bksubmitrefreshallservices>, action_result: false", :backtrace=>nil}
[2023-12-21T09:18:31,362][INFO ][logstash.inputs.jdbc     ][bkprepaidgetserviceassurancetickets] ECS compatibility is enabled but `target` option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the `target` option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)

So i tried to comment out the rest of the pipelines(pipelines.yml) and only enable these 2 to check if they have a configuration problem but they are running perfectly fine :

[2023-12-21T09:29:26,475][WARN ][logstash.javapipeline    ][bkcheckreconnecttries] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
[2023-12-21T09:29:26,475][WARN ][logstash.javapipeline    ][bksubmitrefreshallservices] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
[2023-12-21T09:29:26,486][INFO ][logstash.outputs.elasticsearch][bkcheckreconnecttries] Using a default mapping template {:es_version=>8, :ecs_compatibility=>:v8}
[2023-12-21T09:29:26,487][INFO ][logstash.outputs.elasticsearch][bksubmitrefreshallservices] Using a default mapping template {:es_version=>8, :ecs_compatibility=>:v8}
[2023-12-21T09:29:26,498][INFO ][logstash.javapipeline    ][bkcheckreconnecttries] Starting pipeline {:pipeline_id=>"bkcheckreconnecttries", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["D:/ELK 8.10.2/logstash-8.10.2/config/conf.d/bk_services/bkcheckreconnecttries_config.conf"], :thread=>"#<Thread:0x67e0027c D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2023-12-21T09:29:26,498][INFO ][logstash.javapipeline    ][bksubmitrefreshallservices] Starting pipeline {:pipeline_id=>"bksubmitrefreshallservices", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["D:/ELK 8.10.2/logstash-8.10.2/config/conf.d/bk_services/bksubmitrefreshallservices_config.conf"], :thread=>"#<Thread:0xa536045 D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2023-12-21T09:29:27,250][INFO ][logstash.javapipeline    ][bksubmitrefreshallservices] Pipeline Java execution initialization time {"seconds"=>0.75}
[2023-12-21T09:29:27,250][INFO ][logstash.javapipeline    ][bkcheckreconnecttries] Pipeline Java execution initialization time {"seconds"=>0.75}
[2023-12-21T09:29:28,770][INFO ][logstash.inputs.jdbc     ][bksubmitrefreshallservices] ECS compatibility is enabled but `target` option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the `target` option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)
[2023-12-21T09:29:28,770][INFO ][logstash.inputs.jdbc     ][bkcheckreconnecttries] ECS compatibility is enabled but `target` option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the `target` option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)
[2023-12-21T09:29:28,771][INFO ][logstash.javapipeline    ][bksubmitrefreshallservices] Pipeline started {"pipeline.id"=>"bksubmitrefreshallservices"}
[2023-12-21T09:29:28,771][INFO ][logstash.javapipeline    ][bkcheckreconnecttries] Pipeline started {"pipeline.id"=>"bkcheckreconnecttries"}
[2023-12-21T09:29:28,788][INFO ][logstash.agent           ] Pipelines running {:count=>2, :running_pipelines=>[:bksubmitrefreshallservices, :bkcheckreconnecttries], :non_running_pipelines=>[]}
[2023-12-21T09:30:01,328][INFO ][logstash.inputs.jdbc     ][bksubmitrefreshallservices][8c937ce659684cd3d8ab2f54a70df01bf0eeffb6219744aa5ef3beef436b1e2a] (0.438844s) 

............(couple of lines below : )


[2023-12-21T09:40:00,990][INFO ][logstash.inputs.jdbc     ][bkcheckreconnecttries][640ebbd9aa19e86446b00e227e91d5360b33c5636f6d1b8577f047baa7d5dbc5] (0.004149s) SELECT NULL FROM DUAL
[2023-12-21T09:40:00,990][INFO ][logstash.inputs.jdbc     ][bksubmitrefreshallservices][8c937ce659684cd3d8ab2f54a70df01bf0eeffb6219744aa5ef3beef436b1e2a] (0.003951s) SELECT NULL FROM DUAL
[2023-12-21T09:40:01,221][INFO ][logstash.inputs.jdbc     ][bksubmitrefreshallservices][8c937ce659684cd3d8ab2f54a70df01bf0eeffb6219744aa5ef3beef436b1e2a] (0.229884s)

I also suspect that the number of pipelines terminated may vary after each restart, and usually i dont see the log line :

Pipelines running {:count=>2, :running_pipelines=>[:bksubmitrefreshallservices, :bkcheckreconnecttries]

Do you know another way i can check running pipelines at any given time? I read something about "Pipeline Viewer UI" but i cant seem to find it in kibana, is it a separate software i need to download?

SamehSaeed · December 25, 2023, 1:10pm

I noticed a warning that only appears when using command line, right before the error and pipeline termination :

D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager
D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:225: warning: already initialized constant Sequel::JDBC::JavaSQL::DriverManager

then the error happens ==>

[2023-12-25T15:35:24,053][ERROR][logstash.javapipeline    ][bkgetcustomersimdetails] Pipeline error {:pipeline_id=>"bkgetcustomersimdetails", :exception=>#<NameError: uninitialized constant Sequel::JDBC::Oracle::DatabaseMethods
Did you mean?  Sequel::JDBC::Database>, :backtrace=>["org/jruby/RubyModule.java:4309:in `const_missing'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc/oracle.rb:11:in `block in JDBC'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:379:in `adapter_initialize'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/misc.rb:169:in `initialize'", "org/jruby/RubyClass.java:904:in `new'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/connecting.rb:57:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/core.rb:124:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:123:in `block in jdbc_connect'", "org/jruby/RubyKernel.java:1586:in `loop'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:120:in `jdbc_connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:163:in `open_jdbc_connection'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/inputs/jdbc.rb:309:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-mixin-ecs_compatibility_support-1.3.0-java/lib/logstash/plugin_mixins/ecs_compatibility_support/target_check.rb:48:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:237:in `block in register_plugins'", "org/jruby/RubyArray.java:1987:in `each'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:236:in `register_plugins'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:395:in `start_inputs'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:320:in `start_workers'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:194:in `run'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:146:in `block in start'"], "pipeline.sources"=>["D:/ELK 8.10.2/logstash-8.10.2/config/conf.d/bk_services/bkgetcustomersimdetails_config.conf"], :thread=>"#<Thread:0x3e6a5f0 D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2023-12-25T15:35:23,884][ERROR][logstash.javapipeline    ][bkgetivrguiwavespath] Pipeline error {:pipeline_id=>"bkgetivrguiwavespath", :exception=>#<NameError: uninitialized constant Sequel::JDBC::Oracle::DatabaseMethods
Did you mean?  Sequel::JDBC::Database>, :backtrace=>["org/jruby/RubyModule.java:4309:in `const_missing'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc/oracle.rb:11:in `block in JDBC'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:379:in `adapter_initialize'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/misc.rb:169:in `initialize'", "org/jruby/RubyClass.java:904:in `new'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/connecting.rb:57:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/core.rb:124:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:123:in `block in jdbc_connect'", "org/jruby/RubyKernel.java:1586:in `loop'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:120:in `jdbc_connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:163:in `open_jdbc_connection'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/inputs/jdbc.rb:309:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-mixin-ecs_compatibility_support-1.3.0-java/lib/logstash/plugin_mixins/ecs_compatibility_support/target_check.rb:48:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:237:in `block in register_plugins'", "org/jruby/RubyArray.java:1987:in `each'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:236:in `register_plugins'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:395:in `start_inputs'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:320:in `start_workers'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:194:in `run'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:146:in `block in start'"], "pipeline.sources"=>["D:/ELK 8.10.2/logstash-8.10.2/config/conf.d/bk_services/bkgetivrguiwavespath_config.conf"], :thread=>"#<Thread:0x68437c59 D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2023-12-25T15:35:24,053][ERROR][logstash.javapipeline    ][bksimnumbercallingvalidator] Pipeline error {:pipeline_id=>"bksimnumbercallingvalidator", :exception=>#<NameError: uninitialized constant Sequel::JDBC::Oracle::DatabaseMethods
Did you mean?  Sequel::JDBC::Database>, :backtrace=>["org/jruby/RubyModule.java:4309:in `const_missing'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc/oracle.rb:11:in `block in JDBC'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:379:in `adapter_initialize'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/misc.rb:169:in `initialize'", "org/jruby/RubyClass.java:904:in `new'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/connecting.rb:57:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/core.rb:124:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:123:in `block in jdbc_connect'", "org/jruby/RubyKernel.java:1586:in `loop'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:120:in `jdbc_connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:163:in `open_jdbc_connection'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/inputs/jdbc.rb:309:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-mixin-ecs_compatibility_support-1.3.0-java/lib/logstash/plugin_mixins/ecs_compatibility_support/target_check.rb:48:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:237:in `block in register_plugins'", "org/jruby/RubyArray.java:1987:in `each'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:236:in `register_plugins'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:395:in `start_inputs'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:320:in `start_workers'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:194:in `run'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:146:in `block in start'"], "pipeline.sources"=>["D:/ELK 8.10.2/logstash-8.10.2/config/conf.d/bk_services/bksimnumbercallingvalidator_config.conf"], :thread=>"#<Thread:0x1c5c1465 D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2023-12-25T15:35:24,053][ERROR][logstash.javapipeline    ][bkgetcancellationfees] Pipeline error {:pipeline_id=>"bkgetcancellationfees", :exception=>#<NameError: uninitialized constant Sequel::JDBC::Oracle::DatabaseMethods
Did you mean?  Sequel::JDBC::Database>, :backtrace=>["org/jruby/RubyModule.java:4309:in `const_missing'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc/oracle.rb:11:in `block in JDBC'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/adapters/jdbc.rb:379:in `adapter_initialize'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/misc.rb:169:in `initialize'", "org/jruby/RubyClass.java:904:in `new'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/database/connecting.rb:57:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/sequel-5.71.0/lib/sequel/core.rb:124:in `connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:123:in `block in jdbc_connect'", "org/jruby/RubyKernel.java:1586:in `loop'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:120:in `jdbc_connect'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:163:in `open_jdbc_connection'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.4/lib/logstash/inputs/jdbc.rb:309:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/vendor/bundle/jruby/3.1.0/gems/logstash-mixin-ecs_compatibility_support-1.3.0-java/lib/logstash/plugin_mixins/ecs_compatibility_support/target_check.rb:48:in `register'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:237:in `block in register_plugins'", "org/jruby/RubyArray.java:1987:in `each'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:236:in `register_plugins'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:395:in `start_inputs'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:320:in `start_workers'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:194:in `run'", "D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:146:in `block in start'"], "pipeline.sources"=>["D:/ELK 8.10.2/logstash-8.10.2/config/conf.d/bk_services/bkgetcancellationfees_config.conf"], :thread=>"#<Thread:0x510819a4 D:/ELK 8.10.2/logstash-8.10.2/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2023-12-25T15:35:26,995][INFO ][logstash.javapipeline    ][bksimnumbercallingvalidator] Pipeline terminated {"pipeline.id"=>"bksimnumbercallingvalidator"}
[2023-12-25T15:35:26,993][INFO ][logstash.javapipeline    ][bkgetcustomersimdetails] Pipeline terminated {"pipeline.id"=>"bkgetcustomersimdetails"}
[2023-12-25T15:35:26,998][INFO ][logstash.javapipeline    ][bkgetcancellationfees] Pipeline terminated {"pipeline.id"=>"bkgetcancellationfees"}
[2023-12-25T15:35:26,998][INFO ][logstash.javapipeline    ][bkgetivrguiwavespath] Pipeline terminated {"pipeline.id"=>"bkgetivrguiwavespath"}
[2023-12-25T15:35:27,025][ERROR][logstash.agent           ] Failed to execute action {:id=>:bkgetcustomersimdetails, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<bkgetcustomersimdetails>, action_result: false", :backtrace=>nil}
[2023-12-25T15:35:27,026][ERROR][logstash.agent           ] Failed to execute action {:id=>:bkgetivrguiwavespath, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<bkgetivrguiwavespath>, action_result: false", :backtrace=>nil}
[2023-12-25T15:35:27,026][ERROR][logstash.agent           ] Failed to execute action {:id=>:bksimnumbercallingvalidator, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<bksimnumbercallingvalidator>, action_result: false", :backtrace=>nil}
[2023-12-25T15:35:27,026][ERROR][logstash.agent           ] Failed to execute action {:id=>:bkgetcancellationfees, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<bkgetcancellationfees>, action_result: false", :backtrace=>nil}

Also, there are no useful information about the maximum number of pipelines per node, dont know if this is a driver issue or lack of resources.

system · January 22, 2024, 1:10pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Logstash 6.2.2 with JDBC data on not updated Logstash	6	918	May 4, 2018
Pipeline is being aborted due to logstash_jdbc_last_run File Not Found Logstash docker	3	1159	August 4, 2020
NameError: uninitialized constant Concurrent::ImmediateExecutor Logstash	1	506	December 21, 2018
Error while running multiple pipelines in logstash on centOS 7 Logstash	2	430	July 4, 2019
Logstash intermittently fails to start using jdbc-static filter Logstash	1	342	October 2, 2020

Logstash terminating pipelines error "const_missing, block in JDBC"

Related topics