I am trying to setup a JDBC input plugin.
I have turned off paging :
jdbc_paging_enabled => false
jdbc_page_size => 0
Yet, in the "debug" logs I can see
SELECT count(*) "COUNT" FROM ( [my_query] ) "T1" FETCH NEXT 1 ROWS ONLY
This is a problem for me, since the query will never return the same thing twice.
Since Logstash does not work in cluster, we are trying to "split" the work at the one place all instances are accessing : the DB. We built a query that will return a different chunk everytime, and paging is handled in the DB size. Since each query will return a different result, you can understand why the extra "count" query is problematic
The goal is to indexing X millions records, splitting the load across logstash nodes, and don't index anything twice.
Is there a way to disable this extra "count" query?