Sql_last_value resets to 0 if logstash restart

Elastic Stack Logstash
1

I'm collecting data from jdbc (MS SQL) with this configuration:

jdbc {
    jdbc_driver_library => "C:\dev\script\config\mssql-jdbc-6.2.2.jre8.jar"
    jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
    jdbc_connection_string => "jdbc:sqlserver://1.2.3.4:1433;databaseName=MyDB;"
    jdbc_user => "user"
    jdbc_password => "****"
    jdbc_validate_connection => true
    schedule => "/3 * * * * *"
    statement_filepath => "C:\dev\script\sql\script_es.sql"
    type => "jdbc"
    tags => ["tag1"]
    tracking_column => "createddate"
    use_column_value => true
    clean_run => true  
    last_run_metadata_path => "C:\dev\script\es_jdbc_last_run_metadata.log"
  }

The data collected, and tracking_column works fine. However when I restart the logstash, or change config file and logstash automatically reload (I use flag --config.reload.automatic), then the sql_last_value reset from 0 again. Hence, every time logstash needs to be restarted (maybe because we add new jdbc input), then existing data is re-pulled from jdbc.

Am I do something wrong? It seems that on restart, sql_last_value not read from C:\dev\script\es_jdbc_last_run_metadata.log

Thank you

1 Like
2

Found it, need to delete clean_run parameter as per documentation

2 Likes
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.