How to fetch stream data from ES using node.js?


(M Sayyed) #1

I want to display realtime data from ES to my app.
I am using node.js to fetch data, What will be the best option to fetch realtime data from ES?
I tried to example "Stream search results from Elasticsearch" from npm elasticsearch-stream but its not working.


(Ed) #2

Check out scroll

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html


(M Sayyed) #3

Thanks @eperry for the reply.
I checked scroll but its different than my requirement.
It says Scrolling is not intended for real time user requests, but rather for processing large amounts of data.
Actually my requirement is in first attempt I got data from ES.
Whenever New data will be available in ES which is not fetched yet that should be fetch.
For example I am inserting tweets inside ES from twitter-stream. I want to display all new (realtime) tweets available in ES automatically Without any request from client.


(Ed) #4

You'll have to keep track of the time from each search on your client level, other then scrol there is no way to "save" your position inherent to ES

You could try the Elastic Search Watcher application.


(system) #5