I am running redhat 7.8 and have done a fresh installation of logstash 7.7
When trying to use the logstash-input-jdbc plugin to read from a database I get a No suitable driver found error.
I was able to set it up to work with filebeat 7.7 and tested it runs fine and outputs file content to stdout.
I need to pull data from a db and so I have tried to use the logstash-input-jdbc plugin but have not been able to make it work.
I am using the postgresql-42.2.12.jar driver and have placed it in /usr/share/logstash/logstash-core/lib/jars/ folder
-rw-r--r--. 1 logstash logstash 930498 May 16 16:55 /usr/share/logstash/logstash-core/lib/jars/postgresql-42.2.12.jar
My logstash config:
input {
jdbc {
jdbc_driver_library => "/usr/share/logstash/logstash-core/lib/jars/postgresql-42.2.12.jar"
jdbc_driver_class => "org.postgresql.Driver"
jdbc_connection_string => "jdbc:postgres://host:port/dbname"
jdbc_user => "dbuser"
jdbc_password => "dbpasswd"
use_column_value => true
tracking_column => "id"
statement => "SELECT * FROM table1"
}
}
output {
stdout { codec => rubydebug }
}
I have also tried with removing the jdbc_driver_library entry hoping that the driver would be found in the jars folder but that did not work as well.
Here are the relevant error messages from the log.
[7023]: [2020-05-16T17:06:50,767][ERROR][logstash.inputs.jdbc ][main][886422222895922551f03a9c8e9ec5206a90a9e50a74261bc810b9883509756a] Unable to connect to database. Tried 1 times {:error_message=>"Java::JavaSql::SQLException: No suitable driver found for jdbc:postgres://host:port/dbname"}
[7023]: [2020-05-16T17:06:50,774][ERROR][logstash.javapipeline ][main][886422222895922551f03a9c8e9ec5206a90a9e50a74261bc810b9883509756a] A plugin had an unrecoverable error. Will restart this plugin.
[7023]: Pipeline_id:main
[7023]: Plugin: <LogStash::Inputs::Jdbc jdbc_user=>"dbuser", use_column_value=>true, tracking_column=>"id", jdbc_password=><password>, statement=>"SELECT * FROM table1", jdbc_driver_library=>"/usr/share/logstash/logstash-core/lib/jars/postgresql-42.2.12.jar", jdbc_connection_string=>"jdbc:postgres://host:port/dbname", id=>"886422222895922551f03a9c8e9ec5206a90a9e50a74261bc810b9883509756a", jdbc_driver_class=>"org.postgresql.Driver", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_f3ad5863-ae3b-4c8e-acdb-188047f86ad7", enable_metric=>true, charset=>"UTF-8">, jdbc_paging_enabled=>false, jdbc_page_size=>100000, jdbc_validate_connection=>false, jdbc_validation_timeout=>3600, jdbc_pool_timeout=>5, sql_log_level=>"info", connection_retry_attempts=>1, connection_retry_attempts_wait_time=>0.5, plugin_timezone=>"utc", last_run_metadata_path=>"/usr/share/logstash/.logstash_jdbc_last_run", tracking_column_type=>"numeric", record_last_run=>true, lowercase_column_names=>true, use_prepared_statements=>false>
[7023]: Error: Java::JavaSql::SQLException: No suitable driver found for jdbc:postgres://host:port/dbname
[7023]: Exception: Sequel::DatabaseConnectionError
[7023]: Stack: java.sql.DriverManager.getConnection(java/sql/DriverManager.java:689)
[7023]: java.sql.DriverManager.getConnection(java/sql/DriverManager.java:247)
[7023]: java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)
[7023]: org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:426)
[7023]: org.jruby.javasupport.JavaMethod.invokeStaticDirect(org/jruby/javasupport/JavaMethod.java:358)
[7023]: usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_30_dot_0.lib.sequel.adapters.jdbc.connect(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.30.0/lib/sequel/adapters/jdbc.rb:215)
I tried it on another machine running fedora 31 and logstash 7.7 and tried it from the command line as well but with the same result.
What is the correct way to get logstash to use the jdbc jar file?