JDBC Input (MSSQL) to CSV output to slow - memory no way near Xmx

[romariol (romario)]

Hello everyone,

I'm having a use case where I have a JDBC input to a MSSQL server, fetching all the rows and writing them to a CSV file. the table is quite big (+/- 90 million records) so getting the data and writing it to a csv file without setting any parameters should results in a high memory usage or even out of memory. However, my JVM is stable at +/- 900mb while I would expect it to be near 3gb (my Xmx). No matter what I do, it looks like that I can't increase the throughput.
I have both used the sqljdbc42 and the mssql-jdbc-7.2.2.jre8.jar. No success with both. My config looks like this:

input {
  jdbc {
    jdbc_driver_library => "../driver/sqljdbc42.jar"
    jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
    jdbc_connection_string => "jdbc:sqlserver://test\test2;database=testdb;user=testuser;password=testpassword"
	jdbc_user => nil
    statement => "SELECT * from [dbo].[xxx] WHERE id > :sql_last_value"
    use_column_value => true
    tracking_column => "id"	
	schedule => "52 8 * * *"
    }
}

output {
    csv {
		fields => ["field1","field2","field3"]
		path => "C:\output.csv"
    }
}

Can someone please help me out?

Thanks