Configure logstash input jdbc plugin to disable autocommit for postgresql connection

I've come across the issue where the fetch size cannot be applied for postgresql jdbc connections since the connections default to autocommit, as reported here:

Looking at the documentation I see that under the sequel_opts there is an option to specify an after_connect callable object.

I'm trying to test whether implementing after_connect to disable autocommit will then allow the fetch size to be honored. Or perhaps the after_connect needs to disable autocommit and then reapply the fetch size value.

Does this seems like an approach that may work, or has anyone already gone down this path and seen it fail?

I'm trying to implement this inline within my logstash configuration file. I've been unable to come up with valid Ruby syntax to have the script correctly called.

I've tried several variations -- see below for my latest attempt. I would appreciate any guidance on how to correctly represent this code.

indent preformatted text by 4 spaces
jdbc {
    # Postgres jdbc connection string to our database
    jdbc_connection_string =>  "${JDBC_URL}"
	
    # validate connection before use
    jdbc_validate_connection => true
    
    # The user we wish to execute our statement as
    jdbc_user => "${JDBC_USER}"
	
    # The password for jdbc_user
    jdbc_password => "${JDBC_PWRD}"
			
    # The path to our downloaded jdbc driver
    jdbc_driver_library =>  "${JDBC_JAR}"
	
  
    # The name of the driver class for Postgresql
    jdbc_driver_class =>  "${JDBC_DRIVER}"
	
      
    sql_log_level => "debug"
    jdbc_fetch_size => 10000
	
    sequel_opts => {
	after_connect =>
		"proc { |c| c.autocommit = false } end"				
    }
		
	
	
# our query
    statement => "some valid select statement"   

}

Thank you.
David

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.