I have tried to ingest 8 records data from database..but after some times the rows gets replicated.it is duplicated like 16,24,32...perioodically..how to clear this?

Elastic Stack Logstash
1

input{
jdbc{
clean_run => true
jdbc_driver_library => "C:/Program Files/SQL Developer 4/jdbc/lib/ojdbc8.jar"
jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
jdbc_connection_string => "some connection string"
jdbc_user => "root"
jdbc_password => "root"
statement => "select * from weatherd WHERE low_temp > :sql_last_value"
use_column_value => true
tracking_column => "low_temp"
schedule => "*/5 * * * *"
}
}
output{
elasticsearch{
hosts =>"http://localhost:9200"
index =>"weather1"
}
stdout{}
}

2

Hi,

I have had this problem too a while ago: You are missing the parameter last_run_metadata_path: https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html#_state

This must point to a file which will then contain the last value.

Best regards
Wolfram

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.