# ElasticSearch Stream NEW INCOMING documents to clients

**URL:** <https://discuss.elastic.co/t/elasticsearch-stream-new-incoming-documents-to-clients/261763>\
**Category:** Elasticsearch\
**Created:** [January 21, 2021, 10:15am UTC](https://discuss.elastic.co/t/elasticsearch-stream-new-incoming-documents-to-clients/261763 "2021-01-21T10:15:53Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![schubec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/schubec/32/82658_2.png) [@schubec](https://discuss.elastic.co/u/schubec)\
**Post date:** [January 21, 2021, 10:15am UTC](https://discuss.elastic.co/t/elasticsearch-stream-new-incoming-documents-to-clients/261763/1 "2021-01-21T10:15:53Z")

</div>

Hi!

I have this usecase:

I receive ca 500 documents of cars (brand, model, color, airbag yes/no, etc.) **per second** and I am storing these in ElasticSearch. It works perfectly.

Then, I have a webclient, which searches for, example, all yellow BMW or blue Mercedes cars with airbags. This also works perfectly and in milliseconds.

But to receive the new cars, the user has to hit refresh. I want the results to popup immediately, so **streaming** the result to the users browser (via websockets).

Therefore my server logic (java at the moment, but I can switch to whatever I want) would have to **poll ElasticSearch** every second or so. **This does not seem like a good solution to me.**

Also, I do not have only one client but maybe thousands, so I am concerned about scaling.

So is there any way of streaming the results to the end user by taking into account, that new documents come in all the time.

If this is not possible with ElasticSearch, do you suggest something else? Apache Kafka for example is excellent at streaming, but (as far as I know), not with filtering the results.

PS / Disclaimer: I also asked this question at StackOverflow ( [https://stackoverflow.com/questions/65809522/elasticsearch-stream-new-incoming-documents-to-clients](https://stackoverflow.com/questions/65809522/elasticsearch-stream-new-incoming-documents-to-clients) ) , but then I thought I better ask here, where the ElasticSearch experts are.

Thank you!

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [January 21, 2021, 10:46am UTC](https://discuss.elastic.co/t/elasticsearch-stream-new-incoming-documents-to-clients/261763/2 "2021-01-21T10:46:41Z")

</div>

It sounds like you want [https://github.com/elastic/elasticsearch/issues/1242](https://github.com/elastic/elasticsearch/issues/1242).

---

<div class="post-metadata">

**Author:** ![schubec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/schubec/32/82658_2.png) [@schubec](https://discuss.elastic.co/u/schubec)\
**Post date:** [January 21, 2021, 12:01pm UTC](https://discuss.elastic.co/t/elasticsearch-stream-new-incoming-documents-to-clients/261763/3 "2021-01-21T12:01:20Z")

</div>

Hi David,

yes, you are correct, it seems like the linked feature request.  
But the request is from 2011 with many many "+1" comments, but it seems like it was never implemented.  
So rephrasing it, I would translate it with "We know what you need, but ES cannot offer it at the moment".  
This is not really what I was hoping for, but at least now I have a definitive answer and I have to look for something else.

So thank you for your reply!  
Bernhard

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [January 21, 2021, 12:19pm UTC](https://discuss.elastic.co/t/elasticsearch-stream-new-incoming-documents-to-clients/261763/4 "2021-01-21T12:19:29Z")

</div>

That's right. It's something we'd very much like to implement, which is why it's still open after all these years. We are working towards it, but it's a pretty enormous piece of work.

---

<div class="post-metadata">

**Author:** ![schubec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/schubec/32/82658_2.png) [@schubec](https://discuss.elastic.co/u/schubec)\
**Post date:** [January 21, 2021, 12:35pm UTC](https://discuss.elastic.co/t/elasticsearch-stream-new-incoming-documents-to-clients/261763/5 "2021-01-21T12:35:04Z")

</div>

Yes, that I can imagine!

> We are working towards it

Do you have a timeframe? I can easily wait 2-3 months, but not 2-3 years.

Thank you!

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [January 23, 2021, 11:25am UTC](https://discuss.elastic.co/t/elasticsearch-stream-new-incoming-documents-to-clients/261763/6 "2021-01-23T11:25:09Z")

</div>

No we have no definite timeframe for implementing this feature. It's certainly more than a few months work.

---

<div class="post-metadata">

**Author:** ![schubec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/schubec/32/82658_2.png) [@schubec](https://discuss.elastic.co/u/schubec)\
**Post date:** [January 23, 2021, 1:18pm UTC](https://discuss.elastic.co/t/elasticsearch-stream-new-incoming-documents-to-clients/261763/7 "2021-01-23T13:18:16Z")

</div>

Okay, thank you! Good to know!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 20, 2021, 1:18pm UTC](https://discuss.elastic.co/t/elasticsearch-stream-new-incoming-documents-to-clients/261763/8 "2021-02-20T13:18:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
