# JDBC duplicate records sending to SumoLogic

**URL:** <https://discuss.elastic.co/t/jdbc-duplicate-records-sending-to-sumologic/82950>\
**Category:** Logstash\
**Created:** [April 19, 2017, 7:56pm UTC](https://discuss.elastic.co/t/jdbc-duplicate-records-sending-to-sumologic/82950 "2017-04-19T19:56:42Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![duxy007](https://avatars.discourse-cdn.com/v4/letter/d/f6c823/32.png) [@duxy007](https://discuss.elastic.co/u/duxy007)\
**Post date:** [April 19, 2017, 7:56pm UTC](https://discuss.elastic.co/t/jdbc-duplicate-records-sending-to-sumologic/82950/1 "2017-04-19T19:56:42Z")

</div>

Hi, I have been struggling for several days to understand and prevent getting duplicate records sent from JDBC(Oracle) to SumoLogic using the logstash-output-sumologic plugin. I have been searching the forums but all fixes seems to be for elastic and I am not sure are applicable for my scenario. Below is my conf file. You will see I have my DB connection info and my output set to sumologic.

input {  
jdbc {  
jdbc\_validate\_connection =\> true  
jdbc\_connection\_string =\> "jdbc:oracle:thin:@server:9999:dbname"  
jdbc\_user =\> ""  
jdbc\_password =\> ""  
jdbc\_driver\_library =\> "C:\logstash\ojdbc7.jar"  
jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.OracleDriver"  
statement =\> "select t.log\_id as LogId,really log sql statement where t.log\_id \> :sql\_last\_value"  
use\_column\_value =\> true  
tracking\_column =\> LogId  
record\_last\_run =\> true  
last\_run\_metadata\_path =\> "C:\logstash\bin\logstash\_jdbc\_last\_run"  
schedule=\>"\*/5 \* \* \* \*"  
}  
}

output {  
sumologic{  
url=\>"[https://endpoint1.collection.sumologic.com/blah](https://endpoint1.collection.sumologic.com/blah)"  
format =\> "%{@json}"  
}

I am just at a loss because the way the SQL is written it should not be sending any duplicates. Could it have anything to do with the multiple date fields returning from sql? I appreciate any help. Thanks.

---

<div class="post-metadata">

**Author:** ![duxy007](https://avatars.discourse-cdn.com/v4/letter/d/f6c823/32.png) [@duxy007](https://discuss.elastic.co/u/duxy007)\
**Post date:** [April 25, 2017, 1:24pm UTC](https://discuss.elastic.co/t/jdbc-duplicate-records-sending-to-sumologic/82950/2 "2017-04-25T13:24:55Z")

</div>

I figured out why I was getting duplicates. When I was running the query the tracking column was what I was where I was tracking my place for the query, When I ran the query I noticed that the query did not return data sorted by that column. This is what made the duplicates. Sometimes the the last number was not the last true number so it would pull in data again. To fix all I did was add an order by Logid and now the duplicates are gone.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 23, 2017, 1:30pm UTC](https://discuss.elastic.co/t/jdbc-duplicate-records-sending-to-sumologic/82950/3 "2017-05-23T13:30:13Z")

</div>

Closed on May 23, 2017, 1:30pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.