Jdbc input plugin sql_last_value not updating

Elastic Stack Logstash
1

I have multiple jdbc input plugins in logstash writing to different indices in elastic search.

I'm using sql_last_value to send incremental updates. However, the timestamp in last_run_metadata_path is only getting updated where there are updates to the MySQL database. If there are no updates, timestamp in the last_run_metadata_path file is not updating. Is this expected?

I deleted the files and restarted logstash. However, the old timestamps are back. Has anyone seen this?

2

The sqllast_value is only updated when there are changes seen in the database, because the value is update with the last value read from the database (so the greatest value in your tracking column).
https://www.elastic.co/guide/en/logstash/6.2/plugins-inputs-jdbc.html#_state

Deleting the file while logstash is running isn't of use because logstash writes the file again on shutdown to preserve the last state. To start a new you would have to shutdown logstash, delete the file and start logstash again (or use the config option clean_run to ignore the value saved).

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.