Cassandra jdbc drive issue with logstash in docker

Elastic Stack Logstash
1

Hi,
I am running logsstash, elasticsearch, kibana in docker 8.15.0 and cassandra 5.0.
I want to setup logstash input from cassandra with jdbc. I have downloaded jdbc drivers from Cassandra JDBC Driver Download & Connectivity

below is my logstash pipeline conf file

nput {
jdbc {
jdbc_driver_library => "/usr/share/logstash/logstash-core/lib/jars/cassandra-jdbc-driver.jar"
jdbc_driver_class => "com.dbschema.CassandraJdbcDriver"
jdbc_connection_string => "jdbc:cassandra://cassandra:9042/test_qa"
statement => "SELECT * FROM test WHERE date = :sql_last_value"
jdbc_user => "cassandra"
jdbc_page_size => 100
use_column_value => true
tracking_column => "date"
schedule => "* * * * *"
jdbc_validate_connection => true
}
}

output {
stdout {
codec => json_lines
}
elasticsearch {
hosts => ["http://elasticsearch:9200"]
index => "test_qa_%{+YYYY_MM_dd}"
}
}

and getting error
logstash | [2024-09-25T11:23:36,841][ERROR][logstash.javapipeline ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<LogStash::PluginLoadingError: #<Java::JavaLang::ClassNotFoundException: com.dbschema.CassandraJdbcDriver>. Are you sure you've included the correct jdbc driver in :jdbc_driver_library?>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.11/lib/logstash/plugin_mixins/jdbc/common.rb:46:in load_driver'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.11/lib/logstash/inputs/jdbc.rb:307:in register'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-mixin-ecs_compatibility_support-1.3.0-java/lib/logstash/plugin_mixins/ecs_compatibility_support/target_check.rb:48:in register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:237:in block in register_plugins'", "org/jruby/RubyArray.java:1981:in each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:236:in register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:395:in start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:320:in start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:194:in run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:146:in block in start'"], "pipeline.sources"=>["/usr/share/logstash/pipeline/logstash.conf"], :thread=>"#<Thread:0x4fa0c1e2 /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}

also tried with

and jdbc driver from datastax

and getting error

logstash | [2024-09-25T11:31:45,631][ERROR][logstash.javapipeline ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<LogStash::PluginLoadingError: #<Java::JavaLang::ClassNotFoundException: org.apache.cassandra.cql.jdbc.CassandraDriver>. Are you sure you've included the correct jdbc driver in :jdbc_driver_library?>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.11/lib/logstash/plugin_mixins/jdbc/common.rb:46:in load_driver'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-integration-jdbc-5.4.11/lib/logstash/inputs/jdbc.rb:307:in register'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-mixin-ecs_compatibility_support-1.3.0-java/lib/logstash/plugin_mixins/ecs_compatibility_support/target_check.rb:48:in register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:237:in block in register_plugins'", "org/jruby/RubyArray.java:1981:in each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:236:in register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:395:in start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:320:in start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:194:in run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:146:in block in start'"], "pipeline.sources"=>["/usr/share/logstash/pipeline/logstash.conf"], :thread=>"#<Thread:0x15fee1ca /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}

2

Hi @AmritMatti ,

I assume you've already checked the steps below, right?

  1. Use the correct driver class: For Cassandra 5.0, use "com.datastax.oss.driver.api.core.CqlSession".
  2. Mount the driver jar correctly: Ensure that the driver jar is correctly referenced in the Docker container using the -v mount in Docker.
  3. Verify paths inside the container: Check if the driver is present in /usr/share/logstash/logstash-core/lib/jars/.
3

Hi @Alex_Salgado-Elastic
Thanks for response

yes, I checked jar drivers location but not tried with this driver class. Can you please share which jar file do we have to use from Release 4.17.0 · apache/cassandra-java-driver · GitHub

When i extract this tar file there are many jar files there. May be i am not defining correct one in logstash.conf file