# Save the last fetch/execution time in a persistent file

**URL:** <https://discuss.elastic.co/t/save-the-last-fetch-execution-time-in-a-persistent-file/111959>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [December 15, 2017, 11:08am UTC](https://discuss.elastic.co/t/save-the-last-fetch-execution-time-in-a-persistent-file/111959 "2017-12-15T11:08:37Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mguggi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mguggi/32/79070_2.png) [@mguggi](https://discuss.elastic.co/u/mguggi)\
**Post date:** [December 15, 2017, 11:08am UTC](https://discuss.elastic.co/t/save-the-last-fetch-execution-time-in-a-persistent-file/111959/1 "2017-12-15T11:08:37Z")

</div>

I’m developing a metricbeat module which quries data from a table.

As I would like to fetch the data only once, I want store the last fetch time in a persistent store so that I can pull the required data after a planned or unplanned outage.

Is there such a functionality in libbeat/metricbeat 6.1 which I can use for that (like registry file in filebeat)?

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [December 19, 2017, 5:21am UTC](https://discuss.elastic.co/t/save-the-last-fetch-execution-time-in-a-persistent-file/111959/2 "2017-12-19T05:21:14Z")

</div>

There is no out of the box solution for this at the moment as we have different implementations for different use cases at the moment. We plan to unify this as part of [https://github.com/elastic/beats/issues/5755](https://github.com/elastic/beats/issues/5755) This could make it also possible to be used from modules which I think is a pretty interesting idea. As far as I understand you are looking for a disk cache instead of just a memory cache.

Can you share a bit more about the system you are integrating with?

---

<div class="post-metadata">

**Author:** ![mguggi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mguggi/32/79070_2.png) [@mguggi](https://discuss.elastic.co/u/mguggi)\
**Post date:** [December 22, 2017, 10:12am UTC](https://discuss.elastic.co/t/save-the-last-fetch-execution-time-in-a-persistent-file/111959/3 "2017-12-22T10:12:17Z")

</div>

I want develop an [oraclebeat](https://discuss.elastic.co/t/release-resources-when-metricbeat-stops/107542/8?u=mguggi) on basis on metricbeat. In some metricsets I get records from a table which I don't want receive twice if the beat is planned or unplanned restartet.

Yes, you are right I need something like a disk cache. At the moment I use the simple library [diskv](https://github.com/peterbourgon/diskv) for my use case. My first draft is already provided on [github.com](https://github.com/mguggi/oraclebeat).

@ruflin Thanks for providing the link.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [December 27, 2017, 1:42am UTC](https://discuss.elastic.co/t/save-the-last-fetch-execution-time-in-a-persistent-file/111959/4 "2017-12-27T01:42:18Z")

</div>

Very interesting to see that there is golang lib to connect to oracle: [https://github.com/go-goracle/goracle/tree/v2.1.12](https://github.com/go-goracle/goracle/tree/v2.1.12)

For now you are probably best of to use your own implementation until we provide an interface / method you can use.

Let me know if you hit other issues or if I can help. Really looking forward to see this out in the wild.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 24, 2018, 1:42am UTC](https://discuss.elastic.co/t/save-the-last-fetch-execution-time-in-a-persistent-file/111959/5 "2018-01-24T01:42:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
