# How to Configure Logstash to Forward Data When New Data is Available in Elasticsearch?

**URL:** <https://discuss.elastic.co/t/how-to-configure-logstash-to-forward-data-when-new-data-is-available-in-elasticsearch/373151>\
**Category:** Logstash\
**Created:** [January 14, 2025, 3:44am UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-forward-data-when-new-data-is-available-in-elasticsearch/373151 "2025-01-14T03:44:15Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Wei\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wei_li/32/140638_2.png) [@Wei\_Li](https://discuss.elastic.co/u/Wei_Li)\
**Post date:** [January 14, 2025, 3:44am UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-forward-data-when-new-data-is-available-in-elasticsearch/373151/1 "2025-01-14T03:44:15Z")

</div>

Dear All.

I am a beginner and I have the following questions to ask:

Environment: I have a program that fetches data from a network device API every five minutes and stores it into an Elasticsearch index. The data in the index is continuously overwritten, so the index size remains fixed. It takes about three minutes to fetch and fully store the data into the index. After that, I need to forward the data using Logstash via UDP. The data is fetched every 5 minutes and sent over UDP.

Current issue: Since it takes about 3 minutes to write data into the index, my Logstash pipeline only has 1 to 2 minutes to fetch the current data. If the data in the index hasn’t finished overwriting yet and Logstash starts running, it will miss the data. Is there a way to configure Logstash to send data only when new data has been fully written into the index?

---

<div class="post-metadata">

**Author:** ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)\
**Post date:** [January 15, 2025, 4:54pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-forward-data-when-new-data-is-available-in-elasticsearch/373151/2 "2025-01-15T16:54:49Z")

</div>

Hi @Wei_Li,

Welcome! Some more information would be useful to help you.

1. Can you confirm that this is timestamped data?
2. Where are you forwarding the Elasticsearch data to using Logstash?
3. Is there any particular reason you are overwriting the entire index rather than use ILM to delete data based on the timestamp?

You could probably use Logstash and the [Elasticsearch input plugin](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-elasticsearch.html) to pull the data out. It's possible to provide a [CRON schedule](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-elasticsearch.html#_scheduling) and [query](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-elasticsearch.html#plugins-inputs-elasticsearch-query) as part of the plugin configuration to get the data you need.

Hope that helps!