# Logstash creating more rows than the source table

**URL:** <https://discuss.elastic.co/t/logstash-creating-more-rows-than-the-source-table/99054>\
**Category:** Logstash\
**Created:** [August 31, 2017, 8:35pm UTC](https://discuss.elastic.co/t/logstash-creating-more-rows-than-the-source-table/99054 "2017-08-31T20:35:53Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![aliyesami](https://avatars.discourse-cdn.com/v4/letter/a/ecb155/32.png) [@aliyesami](https://discuss.elastic.co/u/aliyesami)\
**Post date:** [August 31, 2017, 8:35pm UTC](https://discuss.elastic.co/t/logstash-creating-more-rows-than-the-source-table/99054/1 "2017-08-31T20:35:53Z")

</div>

I have only 24062 rows in my RDBMS table but logstash jdbc connection (conf file shown below) keep adding documents to the Elasticsearch in endless loop .

> [root@hadoop1 kibana]# curl 'hadoop5:9200/\_cat/indices?v' | grep pa\_lane\_txn  
> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size  
> green open pa\_lane\_txn \_aCoJU2MRQOnW5uneK2V1w 5 1 129271 0 91.7mb 41.6mb

below is the source rdbms table details

> SQL\> show user  
> USER is "PATRON"  
> SQL\> select count(\*) from tab1;
> 
> ## COUNT(\*)
> 
> ```
> 24062
> 
> ```

the logstash config file is

> input {  
> jdbc {  
> jdbc\_validate\_connection =\> true  
> jdbc\_connection\_string =\> "jdbc:oracle:thin:@patronQA:1526/patron"  
> jdbc\_user =\> "patron"  
> jdbc\_password =\> "xxxxx"  
> jdbc\_driver\_library =\> "/home/admin/ojdbc6.jar"  
> jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.OracleDriver"  
> statement =\> "SELECT TXN\_ID,TXN\_PROCESS\_DATE,TRANSP\_ID,TRANSP\_CLASS,EXT\_PLAZA\_ID,EXT\_LANE\_ID,EXT\_LANE\_TYPE\_CODE,EXT\_DATE\_TIME,TRANSP\_CUR\_  
> BAL,AVC\_CLASS,TOLL\_AMT\_CHARGED,TOLL\_AMT\_COLLECTED,TOLL\_AMT\_FULL,CREDIT\_LIST\_UPDATE\_FLAG,TRANS\_SOURCE,REVCLASS\_REV\_CLASS\_CODE,PAYMENT\_METHOD\_CODE,  
> ENT\_PLAZA\_ID,ENT\_DATE\_TIME,ENT\_LANE\_ID,ENT\_LANE\_TYPE\_CODE,AGENCY\_REJECT\_CODE,MSG\_ID,TRANSP\_INTERNAL\_NUM,UFM\_PAYMENT\_CODE,VEH\_LIC\_NUM,STATE\_ID\_COD  
> E,ORIG\_TXN\_ID from TAB1 WHERE TXN\_PROCESS\_DATE \>:sql\_last\_value "  
> schedule =\> "\*/2 \* \* \* \* \*" ## every 2 secs  
> use\_column\_value =\> "true"  
> tracking\_column =\> "TXN\_PROCESS\_DATE"  
> }  
> }  
> output {  
> elasticsearch {  
> action =\> "index"  
> hosts =\> ["hadoop5:9200"]  
> index =\> "pa\_lane\_txn"  
> document\_type =\> "record"  
> workers =\> 1  
> }  
> }

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 1, 2017, 5:15am UTC](https://discuss.elastic.co/t/logstash-creating-more-rows-than-the-source-table/99054/2 "2017-09-01T05:15:22Z")

</div>

So it seems the `WHERE TXN_PROCESS_DATE >:sql_last_value` part of the query isn't working. What do the queries actually look like (parameters and all)? Is it asking the same query over and over? You may have to bump Logstash's log level to get useful logs (or check the query logs on the Oracle side).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 29, 2017, 5:15am UTC](https://discuss.elastic.co/t/logstash-creating-more-rows-than-the-source-table/99054/3 "2017-09-29T05:15:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.