Hi,
I’m using sql_last_value, last_run_metadata_path, tracking_column etc. for tracking the most recent value of a given column of my query. When I restart logstash, the value is updated correct in the tracking file (last_run_metadata_path). But when new records are found in the next run, the tracking file is updated, but the tracking column itself is not…. As a result, I get multiple records ingested.
Configuration:
jdbc_paging_enabled => true
lowercase_column_names => false
clean_run => false
use_column_value => true
tracking_column => "instant"
tracking_column_type => "timestamp"
query (partly):
statement => "
SELECT
…
instant = e.[Instant],
…
FROM e
WHERE instant > :sql_last_value
ORDER BY instant ASC
OFFSET 0 ROWS
FETCH NEXT 10000 ROWS ONLY
"
Does anybody know whats going wrong?
rgds, Chiel