Hi, I'm currently working on getting the data from multiple DBs using the JDBC input plugin. As the config shown below, I have separate jdbc configs for MySQL and MSSQL.
"
input {
jdbc {
jdbc_driver_library =>"/opt/jdbc-drivers/sqljdbc_6.2/enu/mssql-jdbc-6.2.2.jre8.jar"
jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
jdbc_connection_string => "jdbc:sqlserver://DB_SERVER_IP:1433;databasename=test;"
jdbc_user => "SA"
jdbc_password => "password"
schedule => "* * * * "
statement => "SELECT * from table1 WHERE id > :sql_last_value"
use_column_value => true
tracking_column => id
last_run_metadata_path => "/opt/logstash/config/.last_run.mssql"
clean_run => true
}
jdbc {
jdbc_driver_library =>"/opt/jdbc-drivers/mysql-connector-java-5.1.44/mysql-connector-java-5.1.44-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://DB_SERVER_IP:3306/test"
jdbc_user => "root"
schedule => " * * * *"
statement => "SELECT * from table2 where id > :sql_last_value"
use_column_value => true
tracking_column => id
last_run_metadata_path => "/opt/logstash/config/.last_run.mysql"
clean_run => true
}
}
"
When I restarted the logstash, I got the following logs:
"
Oct 18 22:01:18 ip-10-20-20-50.ap-northeast-1.compute.internal docker[3522]: [2017-10-18T22:01:18,064][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>
Oct 18 22:01:18 ip-10-20-20-50.ap-northeast-1.compute.internal docker[3522]: [2017-10-18T22:01:18,196][INFO ][logstash.pipeline ] Pipeline main started
Oct 18 22:01:18 ip-10-20-20-50.ap-northeast-1.compute.internal docker[3522]: [2017-10-18T22:01:18,244][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
"
And the statements never get executed. But it works fine if I have different input sections for connecting to the same db. I'm wondering if anyone here has any ideas for why it doesn't work.
I also found that the logstash jdbc input can run successfully when all the jdbc inputs in the same config file are connecting to the same type of db even when they are pointing to different hosts. But It won't work properly when there are different jdbc inputs pointing to different type of DB. I suspected there might be some potential issues with the loading different drivers in the same logstash instance?
Hope to get reply soon~ Thanks~