I'm developing a real-time news recommendation engine and was hoping to be
able to use ElasticSearch.
The idea is to store tags for each user in a database
("sport","football","tennis","Olympics","recommender systems", ...).
I would add the news articles to elasticsearch and use the tags to
automatically query ES to generate the recommendations.
I still have a few questions:
Is ES able to do this in (near) real-time ?
Is ES able to handle big (huge) queries ? I would like to send all the
tags for the user to ES, each with it's own weight and would like to
receive a list of news articles that match the users interests.
I'm developing a real-time news recommendation engine and was hoping to be
able to use Elasticsearch.
The idea is to store tags for each user in a database
("sport","football","tennis","Olympics","recommender systems", ...).
I would add the news articles to elasticsearch and use the tags to
automatically query ES to generate the recommendations.
I still have a few questions:
Is ES able to do this in (near) real-time ?
Is ES able to handle big (huge) queries ? I would like to send all the
tags for the user to ES, each with it's own weight and would like to
receive a list of news articles that match the users interests.
Thank you Mark for your quick response.
I was thinking of keeping the tags associated with each user (his
interests) in a separate database.
I would only store the news articles in ES.
When the user requests recommended articles I would fetch his tags and
build a query containing all these tags.
Would this be a decent way of doing it?
I was worried that ES couldn’t handle these huge queries (potentially
hundreds of tags) but apparently I was underestimating ES.
Thanks
Op vrijdag 7 februari 2014 11:31:21 UTC+1 schreef Mark Walkom:
Yes to both, it'll depend on your data and the volumes but ES can handle
storing and retrieving the tags.
I'm developing a real-time news recommendation engine and was hoping to
be able to use Elasticsearch.
The idea is to store tags for each user in a database
("sport","football","tennis","Olympics","recommender systems", ...).
I would add the news articles to elasticsearch and use the tags to
automatically query ES to generate the recommendations.
I still have a few questions:
Is ES able to do this in (near) real-time ?
Is ES able to handle big (huge) queries ? I would like to send all the
tags for the user to ES, each with it's own weight and would like to
receive a list of news articles that match the users interests.
Just to be sure, will I be able to build a query containing a list of tags
each with it's own weight like this:
{"sport":0.25,"tennis":0.54862,"recommender systems":0.236,"search
engine":0.59} ?
Then I would like to extract all documents that match as many tags as
possible.
Is the correct way of doing this using boosts with OR ?
Thank you
Op vrijdag 7 februari 2014 13:09:58 UTC+1 schreef Binh Ly:
Have a look at filters, specifically the terms filter. They execute very
fast and are cached too.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.