# Storm EsSpout from read once to read repeatedly

**URL:** <https://discuss.elastic.co/t/storm-esspout-from-read-once-to-read-repeatedly/97444>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [August 17, 2017, 2:46pm UTC](https://discuss.elastic.co/t/storm-esspout-from-read-once-to-read-repeatedly/97444 "2017-08-17T14:46:05Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jeff\_Bolle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeff_bolle/32/52060_2.png) [@Jeff\_Bolle](https://discuss.elastic.co/u/Jeff_Bolle)\
**Post date:** [August 17, 2017, 2:46pm UTC](https://discuss.elastic.co/t/storm-esspout-from-read-once-to-read-repeatedly/97444/1 "2017-08-17T14:46:05Z")

</div>

The existing Apache Storm EsSpout is designed to use a scroll to search through ES once and then exit. It appears to do this very well and quite reliably.  
However, in my application I'd like to be able to repeatedly run the query and streams the results. If there are no documents the Spout should sit idle for a configurable amount of time before checking again.

Does anyone have any advice they can share on extending EsSpout for this functionality? Before diving in I wanted to ask if anyone had explore this extension before and where the pitfalls may lie. Thanks.

---

<div class="post-metadata">

**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\
**Post date:** [August 17, 2017, 5:38pm UTC](https://discuss.elastic.co/t/storm-esspout-from-read-once-to-read-repeatedly/97444/2 "2017-08-17T17:38:18Z")

</div>

This has been [a suggested feature](https://github.com/elastic/elasticsearch-hadoop/issues/326) for the connector but isn't on the road map for anytime soon. You could add feedback to that issue. Additionally, PR's are always welcome if it's something that interests you.

---

<div class="post-metadata">

**Author:** ![Jeff\_Bolle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeff_bolle/32/52060_2.png) [@Jeff\_Bolle](https://discuss.elastic.co/u/Jeff_Bolle)\
**Post date:** [August 17, 2017, 7:30pm UTC](https://discuss.elastic.co/t/storm-esspout-from-read-once-to-read-repeatedly/97444/3 "2017-08-17T19:30:19Z")

</div>

Wow that is an old request. Interesting that its coming from jnioche. I believe he is behind StormCrawler, which makes complete sense why he wanted to see that. I'll have a look and see how involved the changes look.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 14, 2017, 7:30pm UTC](https://discuss.elastic.co/t/storm-esspout-from-read-once-to-read-repeatedly/97444/4 "2017-09-14T19:30:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
