# Getting some documents ahead of my current time that alter the sql\_last\_value

**URL:** <https://discuss.elastic.co/t/getting-some-documents-ahead-of-my-current-time-that-alter-the-sql-last-value/241945>\
**Category:** Logstash\
**Created:** [July 20, 2020, 11:32pm UTC](https://discuss.elastic.co/t/getting-some-documents-ahead-of-my-current-time-that-alter-the-sql-last-value/241945 "2020-07-20T23:32:59Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Incauto](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/incauto/32/60149_2.png) [@Incauto](https://discuss.elastic.co/u/Incauto)\
**Post date:** [July 20, 2020, 11:32pm UTC](https://discuss.elastic.co/t/getting-some-documents-ahead-of-my-current-time-that-alter-the-sql-last-value/241945/1 "2020-07-20T23:32:59Z")

</div>

I get data from a DB every 30 minutes, in the image the current time is 18:50 pm, I should have no data from 18:30 to current time because this data is collected at 19:00 pm

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/8/f89b7a5e1578fb296a9bdafa90e8852b9b430a5e.png)

It seems that some documents are misconfigured in the DB with a timestamp ahead of time, this also set my sql\_last\_value ahead of time and when JDBC is collecting data I only get half of it.

To solve this I was thinking in get the current time with ruby and compare it with the timestamp, if timestamp its greater than the current time, drop the event.

if I drop the event the sql\_last\_value is still recorded in the last run file?

Is there another way, that you may think, to solve this?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 21, 2020, 1:30am UTC](https://discuss.elastic.co/t/getting-some-documents-ahead-of-my-current-time-that-alter-the-sql-last-value/241945/2 "2020-07-21T01:30:13Z")

</div>

> [@Incauto](#):
>
> if I drop the event the sql\_last\_value is still recorded in the last run file?
> 
> Is there another way, that you may think, to solve this?

sql\_last\_value is stored by the jdbc input. It has no way of knowing whether or not you drop the event, so yes, it is still recorded.

I cannot think of a way to solve this right now.

---

<div class="post-metadata">

**Author:** ![Incauto](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/incauto/32/60149_2.png) [@Incauto](https://discuss.elastic.co/u/Incauto)\
**Post date:** [July 21, 2020, 2:28am UTC](https://discuss.elastic.co/t/getting-some-documents-ahead-of-my-current-time-that-alter-the-sql-last-value/241945/3 "2020-07-21T02:28:42Z")

</div>

If Master Badger can't think of a solution, there is no much hope for me.

the only thing that comes to my mind is to alter the last value with a python script that runs every 25 and 55 minutes in an hour and modifies the value of the epoch timestamp....but Im going to get duplicated values

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [July 21, 2020, 4:45am UTC](https://discuss.elastic.co/t/getting-some-documents-ahead-of-my-current-time-that-alter-the-sql-last-value/241945/4 "2020-07-21T04:45:16Z")

</div>

Do you want to exclude the entries from the future entirely or just wait and import them only as soon as “the future is now” for them? If it's the latter, you could just adjust your query condition to `timestamp > :sql_last_value AND timestamp <= NOW()`, couldn't you?

---

<div class="post-metadata">

**Author:** ![Incauto](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/incauto/32/60149_2.png) [@Incauto](https://discuss.elastic.co/u/Incauto)\
**Post date:** [July 21, 2020, 4:53am UTC](https://discuss.elastic.co/t/getting-some-documents-ahead-of-my-current-time-that-alter-the-sql-last-value/241945/5 "2020-07-21T04:53:32Z")

</div>

😲 Great idea Jenni!, Im going to try what you say

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 18, 2020, 4:53am UTC](https://discuss.elastic.co/t/getting-some-documents-ahead-of-my-current-time-that-alter-the-sql-last-value/241945/6 "2020-08-18T04:53:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 18, 2020, 4:53am UTC](https://discuss.elastic.co/t/getting-some-documents-ahead-of-my-current-time-that-alter-the-sql-last-value/241945/7 "2020-08-18T04:53:34Z")

</div>



---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 4, 2022, 8:07am UTC](https://discuss.elastic.co/t/getting-some-documents-ahead-of-my-current-time-that-alter-the-sql-last-value/241945/8 "2022-11-04T08:07:00Z")

</div>


