Sql_last_value reset to 0 at 12 hour

Elastic Stack Logstash
1

Hi,
I am new to ELK stack. I am planning to fetch data from Oracle table to visualise data in Kibana. My table doesn't have primary key column so based timestamp (format: hhmmss eg. 00001) column scheduling to fetch incremental record using "sql_last_value".
The issue what I am facing with sql_last_value as it's value at end of the day will be set with 235959. When the next day starts sql_last_value value should replace with 0 other wise as timestamp again start from 000000. Hence records cannot be retrieved.

      input {
      jdbc { 
        jdbc_connection_string => "jdbc:mysql://localhost:3306/elk"
        # The user we wish to execute our statement as
        jdbc_user => "jesus"
        jdbc_password => "Jesus@12345"
        # The path to our downloaded jdbc driver
        jdbc_driver_library => "path to oracle jar"
        jdbc_driver_class => "oracle driver"
        # our query
        statement => "SELECT *,id,refnum FROM storeData where localTime>:sql_last_value"
        use_column_value => true
        tracking_column => "id"
        last_run_metadata_path => "/media/jesus/HD4/Software/ELK/.logstash_jdbc_last_run"
        schedule => "*/5 * * * * *"
        }
      }
    output {
     stdout { codec => rubydebug }
      elasticsearch {
      "hosts" => "localhost:9200"
      "index" => "test-2"
      "document_type" => "data"
       "document_id" => "%{id}%{refnum}"
      }
    }
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.