Sql_last_value not working properly

i am trying to ingest the records with following timestamps
.but before that I have successfully ingested 100 rec which having same timestamp.so no w i have my sql last_value _as
so when the next run comes the above 3 records are updating successfully. But instead of taking the last timestamp( as the sql_last_value it is assigning the first record ( in the next run the rest of the 2 records are ingesting again and it is continuning in this manner.can u pls tell how to resolve this.my cobfig file is shown below:

clean_run => true
jdbc_driver_library => "C:/Program Files/SQL Developer 4/jdbc/lib/ojdbc8.jar"
jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
jdbc_connection_string => "connection"
jdbc_user => ""
jdbc_password => "pwd"
statement => "select * from bucket_truck where updated_at > :sql_last_value"
tracking_column => "updated_at"
tracking_column_type => "timestamp"
use_column_value => true
schedule => "*/1 * * * *"

hosts =>"http://localhost:9200"
index =>"ma1"
document_id =>"%{pmd_widget_id}"
document_type => "doc"



This looks like the problem I had two years ago: JDBC to ElasticSearch: duplicate records?

tldr: try to put an order_by updated_at to your query

1 Like

thank you .Its working.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.