Logstash JDBC Plugin Cassandra Driver Not Loaded Error

Elastic Stack Logstash
1

Hi All
Can anyone point me to examples on how logstash can read cassandra DB using JDBC . I tried something like this from logstash installed directory

Step 1 : Create config file for logstash (File name :simple-out-conf) . Below are contents for file
input {
jdbc {
jdbc_connection_string => "jdbc:cassandra://localhost:9160/example"
jdbc_user =>"cassandra"
jdbc_driver_library => "$PATH/cassandra-jdbc-2.1.1.jar"
jdbc_driver_class => "org.apache.cassandra.cql.jdbc.CassandraDriver"
statement => "SELECT * FROM example.news"}}

output {stdout { codec => json_lines }}

Step 2: Executed below command on CLI
./logstash -f simple-out.conf

ouput :
Getting below error on executing above statement
Settings: Default filter workers: 4
Error: org.apache.cassandra.cql.jdbc.CassandraDriver not loaded. Are you sure you've included the correct jdbc driver in :jdbc_driver_library?
You may be interested in the '--configtest' flag which you can
use to validate logstash's configuration before you choose
to restart a running system.
Not sure what i'm missing .Any help is greatly appreciated .

2

Unzip /Users/dodlavijaykumar/Documents/workspace/Java_Activity_Project_1/cassandra-jdbc-2.1.1.jar to check if the org.apache.cassandra.cql.jdbc.CassandraDriver class is present or not.

3

Hi Thomas,
Thanks for replying . I can see the class is present in the path i mentioned ..I unzipped and checked it too .Not sure what i'm missing .

4

Hi All, any help on this will be really appreciated .Did anyone tried connecting Canssandra with logstash .

5

Even from the machine where logstash is running ?

6

Thomas,
I have logstash,Elasticsearch and cassandra running on my local machine . The drivers are also on my local machine .

7

Try running logstash with debug + verbose mode enabled and --configtest.

Then, debug the .rb file ! As you can see on the source:

The error is thrown by Sequel. Maybe link (ln -s) your JAR in the same directory than sequel jar.

8

Just saw your mail ..will give a try and let you know how it goes .

9

Tried all the combinations ..still getting error . I see configuration is OK .
./logstash -f simple-out.conf -v --debug --log test.log --verbose --configtest
Sending logstash logs to test.log.
Configuration OK
Configuration OK

Between i tested using mysql jar file to connect MYSQL DB and it worked .Facing problem only with Cassandra DB.

10

Try to use your JAR with a small java class you write.

11

i used my jar files using eclipse and was able to successfully connect to Cassandra DB .
One catch is i added all dependent libraries to build path to make it work .

Do you think i need to include all the dependencies when i call Cassandra JDBC jar via logstash ? Here is the error is got when using Cassandra JDBC jar

{:timestamp=>"2016-01-04T10:44:44.840000-0600", :message=>"Error: org.apache.cassandra.cql.jdbc.CassandraDriver not loaded. Are you sure you've included the correct jdbc driver in :jdbc_driver_library?"}
{:timestamp=>"2016-01-04T10:44:44.842000-0600", :message=>"You may be interested in the '--configtest' flag which you can\nuse to validate logstash's configuration before you choose\nto restart a running system."}

i made sure i included all the dependencies in to folder and added this folder to classpath and then executed it.

12

Finally .. issue is resolved . The problem was with dependencies jars .Instead of using multiple jars i used a single jar with all the dependencies included in it and it worked .
People who want this integration work please use cassandra-jdbc driver available @ http://www.dbschema.com/cassandra-jdbc-driver.html .

After you execute you might see following messages warning/error messages . But with these errors data is getting extracted and i was able to load it in to ES .
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

Best way is to unjar and have missing jar's included so that you will not see above messages .

Thanks Thomas for your support .

13

Hi Vijay,

I am facing same issue and nothing is working for me. Using same jdbc driver as mentioned by you.

if logstash conf as below
input {
jdbc {
jdbc_connection_string => "jdbc:cassandra://localhost:9160/esdemo"
jdbc_user => "cassandra"
jdbc_driver_library => "D:\driver\cassandrajdbc1.1.jar"
jdbc_driver_class => "org.apache.cassandra.cql.jdbc.CassandraDriver"
statement => "SELECT * from esdemo.sampledata"
}
}
output {

	stdout { codec => json }

}

then error is
Error: org.apache.cassandra.cql.jdbc.CassandraDriver not loaded. Are you sure you've included the correct jdbc driver in :jdbc_driver_library?

and if below conf,

input {
jdbc {
jdbc_connection_string => "jdbc:cassandra://localhost:9160/esdemo"
jdbc_user => "cassandra"
jdbc_driver_library => "D:\driver\cassandrajdbc1.1.jar"
jdbc_driver_class => "com.dbschema.CassandraJdbcDriver"
statement => "SELECT * from esdemo.sampledata"
}
}
output {

	stdout { codec => json }

}
then error is
Pipeline aborted due to error {:exception=>#<Sequel::DatabaseConnectionErro
r: Java::ComDatastaxDriverCoreExceptions::NoHostAvailableException: All host(s)
tried for query failed (tried: localhost/127.0.0.1:9160 (com.datastax.driver.cor
e.exceptions.TransportException: [localhost/127.0.0.1] Channel has been closed))

Is anything missed here?
Cassandra version 2.0.5
Logstash version 2.3.3