# How to configure logstash to read only new entries from database but not using "sql\_last\_start"

**URL:** <https://discuss.elastic.co/t/how-to-configure-logstash-to-read-only-new-entries-from-database-but-not-using-sql-last-start/29140>\
**Category:** Logstash\
**Created:** [September 11, 2015, 1:02pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-read-only-new-entries-from-database-but-not-using-sql-last-start/29140 "2015-09-11T13:02:03Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![IgorG](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igorg/32/4725_2.png) [@IgorG](https://discuss.elastic.co/u/IgorG)\
**Post date:** [September 11, 2015, 1:02pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-read-only-new-entries-from-database-but-not-using-sql-last-start/29140/1 "2015-09-11T13:02:03Z")

</div>

Is it possible to query database like: "SELECT \* FROM TABLE\_NAME where id \> : last\_saved\_id"  
and not by using built-in parameter "sql\_last\_start" e.g. "SELECT \* FROM TABLE\_NAME where timestamp \> : sql\_last\_start".  
I don't have timestamp in my table and I want to schedule logstash to have jdbc input and output in elasticsearch.  
Now, I dont want to query whole table, I want to query only new entries from database but I don't know how and where to save the "id" from previous input. This id will be used in next scheduled time. Let's say, table has 1000 entries, and all of them are read by logstash. Meanwhile, there are 100 new entries. So, when next scheduled logstash is triggered to fetch input data from database, only this 100 new entries should be returned in result set, and passed to logstash output.  
Any idea?

---

<div class="post-metadata">

**Author:** ![talevy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/talevy/32/44896_2.png) [@talevy](https://discuss.elastic.co/u/talevy)\
**Post date:** [September 26, 2015, 7:49pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-read-only-new-entries-from-database-but-not-using-sql-last-start/29140/2 "2015-09-26T19:49:41Z")

</div>

recording max and min values of last seen column values is something we may add in the future. This would result in the ability to run such a query where you check against something like `:latest_max_<column_name>`. One may only benefit from this if their ID structure is ascending in value for every new entry and not a random string. Please file an issue in the github page ([https://github.com/logstash-plugins/logstash-input-jdbc](https://github.com/logstash-plugins/logstash-input-jdbc)) for this and we would be happy to start to tackle this feature!

---

<div class="post-metadata">

**Author:** ![tommy\_o](https://avatars.discourse-cdn.com/v4/letter/t/b782af/32.png) [@tommy\_o](https://discuss.elastic.co/u/tommy_o)\
**Post date:** [January 27, 2016, 12:48am UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-read-only-new-entries-from-database-but-not-using-sql-last-start/29140/3 "2016-01-27T00:48:09Z")

</div>

Was this issue ever submitted to github? If not, I'll submit.

I have a very similar issue, where my data is late arriving but the timestamp is the timestamp of the event, not of being written (reporting nodes ship data every hour, so on the hour, I get the last hour of data). This feature would be incredibly helpful for me.

---

<div class="post-metadata">

**Author:** ![talevy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/talevy/32/44896_2.png) [@talevy](https://discuss.elastic.co/u/talevy)\
**Post date:** [August 3, 2016, 8:23pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-read-only-new-entries-from-database-but-not-using-sql-last-start/29140/4 "2016-08-03T20:23:55Z")

</div>

This is now possible with the latest plugin with support for `tracking_column`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:45am UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-read-only-new-entries-from-database-but-not-using-sql-last-start/29140/5 "2017-07-06T04:45:00Z")

</div>


