I've been fighting this issue for days and keep circling back to the same problem:
[2025-01-13T21:24:32,037][ERROR][logstash.javapipeline ][main] Pipeline error {:pipeline_id=>"main", :exception=>#**<Sequel::AdapterNotFound: LoadError: no such file to load -- sequel/adapters/mongodb>**, :backtrace=>["org/jruby/RubyKernel.java:1187:in `require'", "/nms/apps/logstash/vendor/bundle/jruby/3.1.0/gems/sequel-5.85.0/lib/sequel/database/connecting.rb:84:in `load_adapter'", "/nms/apps/logstash/vendor/bundle/jruby/3.1.0/gems/sequel-5.85.0/lib/sequel/database/connecting.rb:17:in `adapter_class'", "/nms/apps/logstash/vendor/bundle/jruby/3.1.0/gems/sequel-5.85.0/lib/sequel/database/connecting.rb:36:in `connect'", "/nms/apps/logstash/vendor/bundle/jruby/3.1.0/gems/sequel-5.85.0/lib/sequel/core.rb:124:in `connect'", "/nms/apps/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.5.0/lib/logstash/plugin_mixins/jdbc_streaming.rb:62:in `prepare_jdbc_connection'", "/nms/apps/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.5.0/lib/logstash/filters/jdbc_streaming.rb:122:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register'", "/nms/apps/logstash/logstash-core/lib/logstash/java_pipeline.rb:246:in `block in register_plugins'", "org/jruby/RubyArray.java:1981:in `each'", "/nms/apps/logstash/logstash-core/lib/logstash/java_pipeline.rb:245:in `register_plugins'", "/nms/apps/logstash/logstash-core/lib/logstash/java_pipeline.rb:628:in `maybe_setup_out_plugins'", "/nms/apps/logstash/logstash-core/lib/logstash/java_pipeline.rb:258:in `start_workers'", "/nms/apps/logstash/logstash-core/lib/logstash/java_pipeline.rb:199:in `run'", "/nms/apps/logstash/logstash-core/lib/logstash/java_pipeline.rb:151:in `block in start'"], "pipeline.sources"=>["/nms/apps/logstash/config/es_cleanRunningJobs.conf"], :thread=>"#<Thread:0x1327e92b /nms/apps/logstash/logstash-core/lib/logstash/java_pipeline.rb:139 run>"}
The config is:
filter {
jdbc_streaming {
jdbc_driver_library => "/nms/apps/logstash/opt/mongoDB/mongojdbc4.8.jar"
jdbc_driver_class => "Java::com.wisecoders.dbschema.mongodb.JdbcDriver"
jdbc_connection_string => "mongodb://******:*******@mongo-test.fqdn.int:27051/pronghorn?replicaSet=MONGODB_STAG&authSource=admin"
jdbc_user => "******"
jdbc_password => "********"
statement => "db.logstashCollection.find({},{'job_id': :job_id});"
parameters => {"job_id" => "job_id"}
target => "updatedJob"
}
}
What I've tried:
I've downloaded several different Mongo JDBC drivers. With each it took some searching to find the right class name to use, but in the end it comes back to this error:
[main] Pipeline error {:pipeline_id=>"main", :exception=>#<Sequel::AdapterNotFound: LoadError: no such file to load -- sequel/adapters/mongodb>, :backtrace=>["org/jruby/RubyKernel.java:1187:in require'",
Looking in the logstash sequel/adapters directory there is not a mongodb.rb file or a mongodb directory, but I've tried installing:
logstash-output-mongodb
logstash-input-mongodb
logstash-integration-jdbc
I also installed the mongo gem in the version of ruby that comes with logstash:
ruby -S gem install mongo
I'm at a loss. I've searched for the same error and only seen 2 other cases where someone had this issue. In both cases they suggested using a different JDBC driver, and I've used the version they suggested, without any luck.
The use case is, I have a flow of events that are coming in and in cases where a field is empty I need to query mongo to get the data. The input and output work great, but I can't for the life of me get a connection to MongoDB. Interestingly, in the same Logstash installation I have pipelines running where I use MongoDB as an input and it's working flawlessly.