# How does last\_run work for S3?

**URL:** <https://discuss.elastic.co/t/how-does-last-run-work-for-s3/183617>\
**Category:** Logstash\
**Created:** [May 30, 2019, 9:54pm UTC](https://discuss.elastic.co/t/how-does-last-run-work-for-s3/183617 "2019-05-30T21:54:56Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)\
**Post date:** [May 30, 2019, 9:54pm UTC](https://discuss.elastic.co/t/how-does-last-run-work-for-s3/183617/1 "2019-05-30T21:54:57Z")

</div>

From what I've seen, the S3 input last\_run is a date. I didn't think you could use an S3 API to process objects stored from a certain time onward. I thought it needed the key of the last object processed as a marker. Is that not correct?

How does the S3 input plugin accurately keep track of where it left off?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 31, 2019, 12:09am UTC](https://discuss.elastic.co/t/how-does-last-run-work-for-s3/183617/2 "2019-05-31T00:09:36Z")

</div>

The s3 input does not have a last\_run option in the current incarnation (not sure about the history here). It uses sincedb\_path to persist data about what it has processed.

---

<div class="post-metadata">

**Author:** ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)\
**Post date:** [June 1, 2019, 5:20pm UTC](https://discuss.elastic.co/t/how-does-last-run-work-for-s3/183617/3 "2019-06-01T17:20:53Z")

</div>

That's what I meant: `sincedb`. I think that file contains a date. How is that able to work with S3? I thought S3 only works with the last key that was processed, not date.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 1, 2019, 8:38pm UTC](https://discuss.elastic.co/t/how-does-last-run-work-for-s3/183617/4 "2019-06-01T20:38:17Z")

</div>

My understanding is that the s3 input fetches every object from the bucket and compares the last\_modified metadata with the sincedb. If watch\_for\_new\_files is set it will do that over and over, which is why you might want one of the backup options and the delete option so that once an object is processed it is no longer fetched over and over again.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 29, 2019, 8:38pm UTC](https://discuss.elastic.co/t/how-does-last-run-work-for-s3/183617/5 "2019-06-29T20:38:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.