Hi,
I am new to ELK stack. I am planning to fetch data from Oracle table to visualise data in Kibana. My table doesn't have primary key column so based timestamp (format: hhmmss eg. 00001) column scheduling to fetch incremental record using "sql_last_value".
The issue what I am facing with sql_last_value as it's value at end of the day will be set with 235959. When the next day starts sql_last_value value should replace with 0 other wise as timestamp again start from 000000. Hence records cannot be retrieved.
input {
jdbc {
jdbc_connection_string => "jdbc:mysql://localhost:3306/elk"
# The user we wish to execute our statement as
jdbc_user => "jesus"
jdbc_password => "Jesus@12345"
# The path to our downloaded jdbc driver
jdbc_driver_library => "path to oracle jar"
jdbc_driver_class => "oracle driver"
# our query
statement => "SELECT *,id,refnum FROM storeData where localTime>:sql_last_value"
use_column_value => true
tracking_column => "id"
last_run_metadata_path => "/media/jesus/HD4/Software/ELK/.logstash_jdbc_last_run"
schedule => "*/5 * * * * *"
}
}
output {
stdout { codec => rubydebug }
elasticsearch {
"hosts" => "localhost:9200"
"index" => "test-2"
"document_type" => "data"
"document_id" => "%{id}%{refnum}"
}
}