We are using Logstash to push data from Oracle database to Elastic. We are using jdbc-input plugin in logstash. We are using tracking column names 'insertts' to control records to be pushed to elastic.
I observed that whenever logstash service is restarted, it start's pushing data from first record to elastic. Could you please help.
What does your Logstash configuration look like? Logstash keeps track of the last processed record by storing it in ~/.logstash_jdbc_last_run (unless you override the location with the jdbc input's last_run_metdata_path). If you shut down Logstash, what's the contents of that file? Also, look in the Logstash log for clues. If it for some reason has problems creating that file I'd expect it to log a message to that effect.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.