# Adding extra data

**URL:** https://discuss.elastic.co/t/adding-extra-data/143854
**Category:** Logstash
**Created:** [August 10, 2018, 10:09am UTC](https://discuss.elastic.co/t/adding-extra-data/143854 "2018-08-10T10:09:16Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![Olha1](https://avatars.discourse-cdn.com/v4/letter/o/9f8e36/32.png) [@Olha1](https://discuss.elastic.co/u/Olha1)
#### Post date: [August 10, 2018, 10:09am UTC](https://discuss.elastic.co/t/adding-extra-data/143854/1 "2018-08-10T10:09:16Z")

</div>

Help me! Why my logstash with this config file adding the same data repeatedly

input {  
jdbc {  
jdbc\_connection\_string =\> "connection\_string"  
jdbc\_user =\> "user"  
jdbc\_password =\> "password"  
jdbc\_driver\_library =\> "driver\_library"  
jdbc\_driver\_class =\> "org.postgresql.Driver"  
statement =\> 'query'  
}  
}  
filter {

}

output {  
elasticsearch {  
hosts =\> "[http://localhost:9200](http://localhost:9200)"  
index =\> index  
document\_id =\> "%{[ID]}"  
document\_type =\> doc  
user =\> elastic  
password =\> password  
}  
}

---

<div class="post-metadata">

### Author: ![waqark3389](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waqark3389/32/34265_2.png) [@waqark3389](https://discuss.elastic.co/u/waqark3389)
#### Post date: [August 10, 2018, 10:42am UTC](https://discuss.elastic.co/t/adding-extra-data/143854/2 "2018-08-10T10:42:24Z")

</div>

Imagine I asked you the same question, would you be able to give an accurate answer with the amount of information provided? Please revise your question.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 10, 2018, 11:09am UTC](https://discuss.elastic.co/t/adding-extra-data/143854/3 "2018-08-10T11:09:17Z")

</div>

Because the query returns the same data repeatedly? Have you looked at what the jdbc input documentation says about state and the `sql_last_value` parameter?

---

<div class="post-metadata">

### Author: ![Olha1](https://avatars.discourse-cdn.com/v4/letter/o/9f8e36/32.png) [@Olha1](https://discuss.elastic.co/u/Olha1)
#### Post date: [August 10, 2018, 11:11am UTC](https://discuss.elastic.co/t/adding-extra-data/143854/4 "2018-08-10T11:11:34Z")

</div>

My logstash with this config file adding the same data repeatedly. It is like: I have 100 rows in database, but with a few time I will have 100...200...300...etc Why logstash adding the same data again and again?

---

<div class="post-metadata">

### Author: ![Olha1](https://avatars.discourse-cdn.com/v4/letter/o/9f8e36/32.png) [@Olha1](https://discuss.elastic.co/u/Olha1)
#### Post date: [August 10, 2018, 11:12am UTC](https://discuss.elastic.co/t/adding-extra-data/143854/5 "2018-08-10T11:12:34Z")

</div>

I saw this parameter, but it dont help my problem

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 10, 2018, 11:24am UTC](https://discuss.elastic.co/t/adding-extra-data/143854/6 "2018-08-10T11:24:22Z")

</div>

You've configured the elasticsearch output to use the contents of the ID column as the document id. If that works as intended I don't see how you can possibly have 200 documents in ES. What document ids do you have, and how do they relate to the database table's ID column?

---

<div class="post-metadata">

### Author: ![Olha1](https://avatars.discourse-cdn.com/v4/letter/o/9f8e36/32.png) [@Olha1](https://discuss.elastic.co/u/Olha1)
#### Post date: [August 10, 2018, 11:46am UTC](https://discuss.elastic.co/t/adding-extra-data/143854/7 "2018-08-10T11:46:24Z")

</div>

If I have this row document\_id =\> "%{[ID]}", then I have updating my data all time.  
If I dont have this row document\_id =\> "%{[ID]}", then I have adding same data in elastic.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 10, 2018, 12:48pm UTC](https://discuss.elastic.co/t/adding-extra-data/143854/8 "2018-08-10T12:48:46Z")

</div>

You're asking the same question as a couple of days ago ([Updating data](https://discuss.elastic.co/t/updating-data/143186/3)). Please continue that thread instead. I didn't see a response to my last post ("If your table has a column ...").

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 7, 2018, 12:55pm UTC](https://discuss.elastic.co/t/adding-extra-data/143854/9 "2018-09-07T12:55:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
