Hello, I am currently doing a proof of concept where I try to pull data from 2 different databases (SQL and Snowflake). I don't see any error in the logs when running the conf file and logstash never seem to complete even though I'm query top 1 record for each database. I am using v5.6 logstash version.
Sample conf:
input {
jdbc {
jdbc_driver_library => "/home/ubuntu/sqljdbc_6.2/enu/mssql-jdbc-6.2.1.jre8.jar"
jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
jdbc_connection_string => "jdbc:sqlserver://<server>;user=<user>;password=<password>;"
jdbc_user => "<user>"
jdbc_password => "<password>"
jdbc_validate_connection => true
jdbc_validation_timeout => -1
last_run_metadata_path => "/etc/logstash/.logstash_jdbc_employee_last_run"
clean_run => true
statement => "SELECT TOP 1 * FROM [ABC].[dbo].[Employee] ORDER BY ID"
type => "employee"
}
jdbc {
jdbc_driver_library => "/home/ubuntu/snowflake-jdbc-3.13.0/snowflake-jdbc-3.13.0.jar"
jdbc_driver_class => "com.snowflake.client.jdbc.SnowflakeDriver"
jdbc_connection_string => "jdbc:snowflake://<account name>.snowflakecomputing.com/?role=<role>&warehouse=<warehouse>&db=<db>&schema=<schema>"
jdbc_user => "<user>"
jdbc_password => "<password>"
jdbc_validate_connection => true
jdbc_validation_timeout => -1
last_run_metadata_path => "/etc/logstash/.logstash_jdbc_department_last_run"
clean_run => true
statement => "SELECT TOP 1 * FROM WH.PUB_COM.V_DEPT"
type => "department"
}
}
output {
}
I didn't see any documentation that says logstash does not support loading multiple types of db is not supported. The closest one I saw is from another forum post similar to what I'm trying to do but never answered.