Scheduling data import into elasticsearch from a api resource?


(AbdulBasit KABIR) #1

Am working on a project where I would like to use elasticsearch to implement the search.
But the data to be searched is gotten from an API path, So how would I index the API resource into elasticsearch and how do I auto schedule it to re-index at some intervals?

For example, I want to index ALL the documents gotten for this path https://api.github.com/gists/public into elasticsearch and I want elasticsearch to automatically check and update any changes to the documents after every 48hrs.

Your help would really be appreciated.


(Magnus B├Ąck) #2

Have a look at Logstash and its http_poller plugin.


(system) #3