# Using river plugin for live requests **URL:** \ **Category:** Elasticsearch\ **Created:** [December 23, 2013, 6:13pm UTC](https://discuss.elastic.co/t/using-river-plugin-for-live-requests/15003 "2013-12-23T18:13:30Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![chh](https://avatars.discourse-cdn.com/v4/letter/c/7ab992/32.png) [@chh](https://discuss.elastic.co/u/chh)\ **Post date:** [December 23, 2013, 6:13pm UTC](https://discuss.elastic.co/t/using-river-plugin-for-live-requests/15003/1 "2013-12-23T18:13:30Z")
Hello, is it possible to use elasticsearch for live river requests? I want to use elasticsearch to collect data from multiple APIs - but only if someone is really searching for it. For example: User searches for a boat available between xmas and newyear - now i want to use the river plugin to fetch the data from all the APIs and merge it into my index Best regards Christian -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/db3b0a57-9261-4a31-8783-94edbbadbd3b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/db3b0a57-9261-4a31-8783-94edbbadbd3b%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Karol\_Gwaj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karol_gwaj/32/1324_2.png) [@Karol\_Gwaj](https://discuss.elastic.co/u/Karol_Gwaj)\ **Post date:** [December 24, 2013, 2:40am UTC](https://discuss.elastic.co/t/using-river-plugin-for-live-requests/15003/2 "2013-12-24T02:40:35Z")
by live request, do you mean the situation that user hits elasticsearch with query, and then waits for elasticsearch river to pull data from some API (before responding back to user)? is so then i dont think the river is the way to go here if in other hand you can populate your index a little bit later, then you can use river for that you can implement custom river that will go through list of most popular queries, run them through API, and index Cheer, On Monday, December 23, 2013 6:13:30 PM UTC, chh wrote: > Hello, > > is it possible to use elasticsearch for live river requests? > I want to use elasticsearch to collect data from multiple APIs - but only > if someone is really searching for it. > > For example: User searches for a boat available between xmas and newyear - > now i want to use the river plugin to fetch the data from all the APIs and > merge it into my index > > Best regards > > Christian -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/63d5c4d7-760c-47d7-b039-423838cee4e6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/63d5c4d7-760c-47d7-b039-423838cee4e6%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [December 25, 2013, 12:14am UTC](https://discuss.elastic.co/t/using-river-plugin-for-live-requests/15003/3 "2013-12-25T00:14:31Z")
Hey, rivers are used to pull data in - but totally independent from incoming search requests. You could write your own plugin, which intercepts a search request and forwards it to your API (and indexes the results afterwards), but it might make more sense to actually execute kind of 'search' in another component of your architecture. So, execute the query, return the results to the requester and store them in your index, so you can use it for searches from then on. Wondering about the use-case here, interested to know more. --Alex On Mon, Dec 23, 2013 at 7:13 PM, chh [chh@miet24.de](mailto:chh@miet24.de) wrote: > Hello, > > is it possible to use elasticsearch for live river requests? > I want to use elasticsearch to collect data from multiple APIs - but only > if someone is really searching for it. > > For example: User searches for a boat available between xmas and newyear - > now i want to use the river plugin to fetch the data from all the APIs and > merge it into my index > > Best regards > > Christian > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/db3b0a57-9261-4a31-8783-94edbbadbd3b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/db3b0a57-9261-4a31-8783-94edbbadbd3b%40googlegroups.com) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM\_r9s0moTN7%3DD8ox11N%2BLjwX%2B8bPxuCKT9nn6YQpOK4bw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_r9s0moTN7%3DD8ox11N%2BLjwX%2B8bPxuCKT9nn6YQpOK4bw%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [December 25, 2013, 4:24pm UTC](https://discuss.elastic.co/t/using-river-plugin-for-live-requests/15003/4 "2013-12-25T16:24:18Z")
It looks like your use case is federated search, the approach that let users search for what they want, but in the background a dispatcher is analyzing the request to forward the search to only relevant sources that might be able to match the request. Rivers are not the instrument for federated search. And there is no solution for that, if you still want performance - the idea is to aggregate all sources and consolidate it into a single ES cluster for the fastest possible search. Of course you can experiment by federating several ES clusters but in that case you must do the heavy lifting (distributed indexing, ranking, faceting, filtering) by your own code. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF09oySa075vgsi1UShzMSj5h5T\_atCN5hbAaZSDTtnsg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF09oySa075vgsi1UShzMSj5h5T_atCN5hbAaZSDTtnsg%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![chh](https://avatars.discourse-cdn.com/v4/letter/c/7ab992/32.png) [@chh](https://discuss.elastic.co/u/chh)\ **Post date:** [December 27, 2013, 8:25am UTC](https://discuss.elastic.co/t/using-river-plugin-for-live-requests/15003/5 "2013-12-27T08:25:53Z")
Good morning, thanks for your answers. I hope everyone had a nice xmas. Like Jörg posted: 'federated search' is what i was trying to accomplish within ES - yet i forgot the name. Would've been nice to have a single place to collect and serve the data. Unfortunately there are too many options to simply index all the foreign data. Christian Am Mittwoch, 25. Dezember 2013 17:24:18 UTC+1 schrieb Jörg Prante: > It looks like your use case is federated search, the approach that let > users search for what they want, but in the background a dispatcher is > analyzing the request to forward the search to only relevant sources that > might be able to match the request. > > Rivers are not the instrument for federated search. And there is no > solution for that, if you still want performance - the idea is to aggregate > all sources and consolidate it into a single ES cluster for the fastest > possible search. > > Of course you can experiment by federating several ES clusters but in that > case you must do the heavy lifting (distributed indexing, ranking, > faceting, filtering) by your own code. > > Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c432d105-dd5a-445c-a19c-d16a958d2470%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c432d105-dd5a-445c-a19c-d16a958d2470%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:59am UTC](https://discuss.elastic.co/t/using-river-plugin-for-live-requests/15003/6 "2017-07-06T01:59:18Z")