# Handling duplicates from SQL Dumps/json and API response in Logstash

**URL:** <https://discuss.elastic.co/t/handling-duplicates-from-sql-dumps-json-and-api-response-in-logstash/154351>\
**Category:** Logstash\
**Created:** [October 28, 2018, 6:31pm UTC](https://discuss.elastic.co/t/handling-duplicates-from-sql-dumps-json-and-api-response-in-logstash/154351 "2018-10-28T18:31:10Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![balumurari1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/balumurari1/32/39203_2.png) [@balumurari1](https://discuss.elastic.co/u/balumurari1)\
**Post date:** [October 29, 2018, 6:55am UTC](https://discuss.elastic.co/t/handling-duplicates-from-sql-dumps-json-and-api-response-in-logstash/154351/3 "2018-10-29T06:55:15Z")

</div>

use scheduler, so that it will update the data for every 1 minute as shown below,

The input code is fine, in the output did you mention elasticsearch to get the result,

Try this,

input {

jdbc {  
jdbc\_driver\_library =\> "xxxx\oracle-10g\ojdbc14.jar"  
jdbc\_driver\_class =\> "oracle.jdbc.driver.OracleDriver"  
jdbc\_connection\_string =\> "jdbc:oracle:thin:@localhost:1521:DATABASE"  
jdbc\_user =\> "ROMAINROM"  
jdbc\_password =\> "ROMAINROM"  
statement =\> "SELECT TOP 10 \* FROM TABLE"  
jdbc\_paging\_enabled =\> "true"  
jdbc\_page\_size =\> "50000"  
schedule =\> "\*/1 \* \* \* \*"  
}

}

output{  
elasticsearch { codec =\> json hosts =\> ["localhost:9200"] index =\> "index9" }  
stdout { codec =\> rubydebug }  
}

---

_[View the full topic](https://discuss.elastic.co/t/handling-duplicates-from-sql-dumps-json-and-api-response-in-logstash/154351)._