# Automatic fetch of data to index **URL:** \ **Category:** Elasticsearch\ **Created:** [September 2, 2011, 8:03am UTC](https://discuss.elastic.co/t/automatic-fetch-of-data-to-index/5301 "2011-09-02T08:03:23Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![powgri](https://avatars.discourse-cdn.com/v4/letter/p/7993a0/32.png) [@powgri](https://discuss.elastic.co/u/powgri)\ **Post date:** [September 2, 2011, 8:03am UTC](https://discuss.elastic.co/t/automatic-fetch-of-data-to-index/5301/1 "2011-09-02T08:03:23Z")
I have a RESTFul webservice and a /store/list endpoint which responds back with the following json: How can I setup an elasticsearch instance to automatically pull data in intervals from this endpoint based on the HTTP response headers set by the service? { "stores": { "@rows": "15", "@total": "2239", "store": [ { "id": 8095, "link": [ { "@href": "[http://dv1-cbsp4.abcd.int:8080/defg/](http://dv1-cbsp4.abcd.int:8080/defg/) 1/store/8095", "@rel": "related", "@title": "store" }, { "@href": "[http://dv1-cbsp4.abcd.int:8080/defg/](http://dv1-cbsp4.abcd.int:8080/defg/) 1/promotion/store/8095/list", "@rel": "related", "@title": "store.promotion.list" }, { "@href": "[http://dv1-cbsp4.abcd.int:8080/defg/](http://dv1-cbsp4.abcd.int:8080/defg/) 1/campaign/store/8095/list", "@rel": "related", "@title": "store.campaign.list" }, { "@href": "[http://dv1-cbsp4.abcd.int:8080/defg/](http://dv1-cbsp4.abcd.int:8080/defg/) 1/store/8095/keyword/list", "@rel": "related", "@title": "store.keyword.list" } ], "name": "Abebooks", "reward": { "calculation": "Percentage", "currency": "USD", "amount": 5, "display": "Percentage" }, "totalReward": { "calculation": "Percentage", "currency": "USD", "amount": 5, "display": "Percentage" }, "desc": "abebooks" }, ... ... ---
**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\ **Post date:** [September 2, 2011, 8:50am UTC](https://discuss.elastic.co/t/automatic-fetch-of-data-to-index/5301/2 "2011-09-02T08:50:21Z")
Create a river based plugin. Thanks Vineeth On Fri, Sep 2, 2011 at 1:33 PM, JJ [powgri@gmail.com](mailto:powgri@gmail.com) wrote: > I have a RESTFul webservice and a /store/list endpoint which responds > back with the following json: > How can I setup an elasticsearch instance to automatically pull data > in intervals from this endpoint based on the HTTP response headers set > by the service? > > { > "stores": { > "@rows": "15", > "@total": "2239", > "store": [ > { > "id": 8095, > "link": [ > { > "@href": "[http://dv1-cbsp4.abcd.int:8080/defg/](http://dv1-cbsp4.abcd.int:8080/defg/) > 1/store/8095", > "@rel": "related", > "@title": "store" > }, > { > "@href": "[http://dv1-cbsp4.abcd.int:8080/defg/](http://dv1-cbsp4.abcd.int:8080/defg/) > 1/promotion/store/8095/list", > "@rel": "related", > "@title": "store.promotion.list" > }, > { > "@href": "[http://dv1-cbsp4.abcd.int:8080/defg/](http://dv1-cbsp4.abcd.int:8080/defg/) > 1/campaign/store/8095/list", > "@rel": "related", > "@title": "store.campaign.list" > }, > { > "@href": "[http://dv1-cbsp4.abcd.int:8080/defg/](http://dv1-cbsp4.abcd.int:8080/defg/) > 1/store/8095/keyword/list", > "@rel": "related", > "@title": "store.keyword.list" > } > ], > "name": "Abebooks", > "reward": { > "calculation": "Percentage", > "currency": "USD", > "amount": 5, > "display": "Percentage" > }, > "totalReward": { > "calculation": "Percentage", > "currency": "USD", > "amount": 5, > "display": "Percentage" > }, > "desc": "abebooks" > }, > ... > ... ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [September 2, 2011, 9:39am UTC](https://discuss.elastic.co/t/automatic-fetch-of-data-to-index/5301/3 "2011-09-02T09:39:54Z")
You can't, you need ot do it yourself. A river (as suggested in another thread) is an option, but it might get too complex for you, a simple process (in your fav lang) that does it is good enough. On Fri, Sep 2, 2011 at 11:03 AM, JJ [powgri@gmail.com](mailto:powgri@gmail.com) wrote: > I have a RESTFul webservice and a /store/list endpoint which responds > back with the following json: > How can I setup an elasticsearch instance to automatically pull data > in intervals from this endpoint based on the HTTP response headers set > by the service? > > { > "stores": { > "@rows": "15", > "@total": "2239", > "store": [ > { > "id": 8095, > "link": [ > { > "@href": "[http://dv1-cbsp4.abcd.int:8080/defg/](http://dv1-cbsp4.abcd.int:8080/defg/) > 1/store/8095", > "@rel": "related", > "@title": "store" > }, > { > "@href": "[http://dv1-cbsp4.abcd.int:8080/defg/](http://dv1-cbsp4.abcd.int:8080/defg/) > 1/promotion/store/8095/list", > "@rel": "related", > "@title": "store.promotion.list" > }, > { > "@href": "[http://dv1-cbsp4.abcd.int:8080/defg/](http://dv1-cbsp4.abcd.int:8080/defg/) > 1/campaign/store/8095/list", > "@rel": "related", > "@title": "store.campaign.list" > }, > { > "@href": "[http://dv1-cbsp4.abcd.int:8080/defg/](http://dv1-cbsp4.abcd.int:8080/defg/) > 1/store/8095/keyword/list", > "@rel": "related", > "@title": "store.keyword.list" > } > ], > "name": "Abebooks", > "reward": { > "calculation": "Percentage", > "currency": "USD", > "amount": 5, > "display": "Percentage" > }, > "totalReward": { > "calculation": "Percentage", > "currency": "USD", > "amount": 5, > "display": "Percentage" > }, > "desc": "abebooks" > }, > ... > ... ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:55am UTC](https://discuss.elastic.co/t/automatic-fetch-of-data-to-index/5301/4 "2017-07-06T03:55:17Z")