I've come across the issue where the fetch size cannot be applied for postgresql jdbc connections since the connections default to autocommit, as reported here:
Looking at the documentation I see that under the sequel_opts there is an option to specify an after_connect callable object.
I'm trying to test whether implementing after_connect to disable autocommit will then allow the fetch size to be honored. Or perhaps the after_connect needs to disable autocommit and then reapply the fetch size value.
Does this seems like an approach that may work, or has anyone already gone down this path and seen it fail?
I'm trying to implement this inline within my logstash configuration file. I've been unable to come up with valid Ruby syntax to have the script correctly called.
I've tried several variations -- see below for my latest attempt. I would appreciate any guidance on how to correctly represent this code.
indent preformatted text by 4 spaces
jdbc {
# Postgres jdbc connection string to our database
jdbc_connection_string => "${JDBC_URL}"
# validate connection before use
jdbc_validate_connection => true
# The user we wish to execute our statement as
jdbc_user => "${JDBC_USER}"
# The password for jdbc_user
jdbc_password => "${JDBC_PWRD}"
# The path to our downloaded jdbc driver
jdbc_driver_library => "${JDBC_JAR}"
# The name of the driver class for Postgresql
jdbc_driver_class => "${JDBC_DRIVER}"
sql_log_level => "debug"
jdbc_fetch_size => 10000
sequel_opts => {
after_connect =>
"proc { |c| c.autocommit = false } end"
}
# our query
statement => "some valid select statement"
}
Thank you.
David