# ElasticSearch as a recommender? **URL:** \ **Category:** Elasticsearch\ **Created:** [February 7, 2014, 10:26am UTC](https://discuss.elastic.co/t/elasticsearch-as-a-recommender/15666 "2014-02-07T10:26:50Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![sam\_leroux](https://avatars.discourse-cdn.com/v4/letter/s/c37758/32.png) [@sam\_leroux](https://discuss.elastic.co/u/sam_leroux)\ **Post date:** [February 7, 2014, 10:26am UTC](https://discuss.elastic.co/t/elasticsearch-as-a-recommender/15666/1 "2014-02-07T10:26:50Z")
Hello, I'm developing a real-time news recommendation engine and was hoping to be able to use ElasticSearch. The idea is to store tags for each user in a database ("sport","football","tennis","Olympics","recommender systems", ...). I would add the news articles to elasticsearch and use the tags to automatically query ES to generate the recommendations. I still have a few questions: - Is ES able to do this in (near) real-time ? - Is ES able to handle big (huge) queries ? I would like to send all the tags for the user to ES, each with it's own weight and would like to receive a list of news articles that match the users interests. Thanks -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0553ccec-66e1-4623-ac3d-380ba9a2c4a1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0553ccec-66e1-4623-ac3d-380ba9a2c4a1%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 7, 2014, 10:31am UTC](https://discuss.elastic.co/t/elasticsearch-as-a-recommender/15666/2 "2014-02-07T10:31:21Z")
Yes to both, it'll depend on your data and the volumes but ES can handle storing and retrieving the tags. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 7 February 2014 21:26, [sam.leroux@gmail.com](mailto:sam.leroux@gmail.com) wrote: > Hello, > > I'm developing a real-time news recommendation engine and was hoping to be > able to use Elasticsearch. > The idea is to store tags for each user in a database > ("sport","football","tennis","Olympics","recommender systems", ...). > I would add the news articles to elasticsearch and use the tags to > automatically query ES to generate the recommendations. > > I still have a few questions: > > - Is ES able to do this in (near) real-time ? > - Is ES able to handle big (huge) queries ? I would like to send all the > tags for the user to ES, each with it's own weight and would like to > receive a list of news articles that match the users interests. > > Thanks > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/0553ccec-66e1-4623-ac3d-380ba9a2c4a1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0553ccec-66e1-4623-ac3d-380ba9a2c4a1%40googlegroups.com) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624ZvwrgJO5ZDGSQXNk4C2-c1Shn2JV%3DcGHf%3DrV5O9sQoXQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624ZvwrgJO5ZDGSQXNk4C2-c1Shn2JV%3DcGHf%3DrV5O9sQoXQ%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![sam\_leroux](https://avatars.discourse-cdn.com/v4/letter/s/c37758/32.png) [@sam\_leroux](https://discuss.elastic.co/u/sam_leroux)\ **Post date:** [February 7, 2014, 10:57am UTC](https://discuss.elastic.co/t/elasticsearch-as-a-recommender/15666/3 "2014-02-07T10:57:15Z")
Thank you Mark for your quick response. I was thinking of keeping the tags associated with each user (his interests) in a separate database. I would only store the news articles in ES. When the user requests recommended articles I would fetch his tags and build a query containing all these tags. Would this be a decent way of doing it? I was worried that ES couldn’t handle these huge queries (potentially hundreds of tags) but apparently I was underestimating ES. Thanks Op vrijdag 7 februari 2014 11:31:21 UTC+1 schreef Mark Walkom: > Yes to both, it'll depend on your data and the volumes but ES can handle > storing and retrieving the tags. > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) \ > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > On 7 February 2014 21:26, \<[sam.l...@gmail.com](mailto:sam.l...@gmail.com) \\> wrote: > > > Hello, > > > > I'm developing a real-time news recommendation engine and was hoping to > > be able to use Elasticsearch. > > The idea is to store tags for each user in a database > > ("sport","football","tennis","Olympics","recommender systems", ...). > > I would add the news articles to elasticsearch and use the tags to > > automatically query ES to generate the recommendations. > > > > I still have a few questions: > > > > - Is ES able to do this in (near) real-time ? > > - Is ES able to handle big (huge) queries ? I would like to send all the > > tags for the user to ES, each with it's own weight and would like to > > receive a list of news articles that match the users interests. > > > > Thanks > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/0553ccec-66e1-4623-ac3d-380ba9a2c4a1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0553ccec-66e1-4623-ac3d-380ba9a2c4a1%40googlegroups.com) > > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1351fc79-051b-4207-bbb3-2174f4d832bc%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1351fc79-051b-4207-bbb3-2174f4d832bc%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Binh\_Ly](https://avatars.discourse-cdn.com/v4/letter/b/ce7236/32.png) [@Binh\_Ly](https://discuss.elastic.co/u/Binh_Ly)\ **Post date:** [February 7, 2014, 12:09pm UTC](https://discuss.elastic.co/t/elasticsearch-as-a-recommender/15666/4 "2014-02-07T12:09:58Z")
Have a look at filters, specifically the terms filter. They execute very fast and are cached too. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. On Friday, February 7, 2014 5:57:15 AM UTC-5, [sam.l...@gmail.com](mailto:sam.l...@gmail.com) wrote: > I was worried that ES couldn’t handle these huge queries (potentially > hundreds of tags) but apparently I was underestimating ES. > > Thanks -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c336837b-eb9c-4845-97dd-331655817dad%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c336837b-eb9c-4845-97dd-331655817dad%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![sam\_leroux](https://avatars.discourse-cdn.com/v4/letter/s/c37758/32.png) [@sam\_leroux](https://discuss.elastic.co/u/sam_leroux)\ **Post date:** [February 8, 2014, 11:27am UTC](https://discuss.elastic.co/t/elasticsearch-as-a-recommender/15666/5 "2014-02-08T11:27:51Z")
Thank you Binh, that looks promising. Just to be sure, will I be able to build a query containing a list of tags each with it's own weight like this: {"sport":0.25,"tennis":0.54862,"recommender systems":0.236,"search engine":0.59} ? Then I would like to extract all documents that match as many tags as possible. Is the correct way of doing this using boosts with OR ? Thank you Op vrijdag 7 februari 2014 13:09:58 UTC+1 schreef Binh Ly: > Have a look at filters, specifically the terms filter. They execute very > fast and are cached too. > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/all-about-elasticsearch-filter-bitsets/) > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html) > > On Friday, February 7, 2014 5:57:15 AM UTC-5, [sam.l...@gmail.com](mailto:sam.l...@gmail.com) wrote: > > > I was worried that ES couldn’t handle these huge queries (potentially > > hundreds of tags) but apparently I was underestimating ES. > > > > Thanks -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0a420fca-7fb7-4640-9cfc-642b7c3df958%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0a420fca-7fb7-4640-9cfc-642b7c3df958%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:51am UTC](https://discuss.elastic.co/t/elasticsearch-as-a-recommender/15666/6 "2017-07-06T01:51:35Z")