We built a video search platform powered by Elasticsearch. Which consists of a search bar (of course) and many interesting functions like video recommendation, search histories, video comments and so on.
Which confuses me is that how to track relative events to increase user experience in that platform --- e.g I need to know if a user view any videos from the top K recalled results after he clicking the "search" button, and if he has any interest from the information presented to him.
It would be a great help to get any suggestion from you!
Hi.
I can say about how we work in the company.
Today we use both GoogleAnalycts and Data Studio to track metrics such as CTR rate, most searched terms, terms without search results and others that depend on the configuration of each case.
In addition, we have developed a service that allows you to generate events from any interaction.
When a user does a search, we trigger an event, when he receives the search results, another event is triggered.
If the user clicks on the result, another event is triggered.
In these events we put data that is relevant so far, such as: session, term, results array, order of results, the position of the selected item.
In this way, we are able to better monitor metrics such as NDCG and MRR, which are the ones we define for now, but from the events we can define more metrics.
After few discussions with colleagues, I made a draft about how to track events for the further search relavence analysis.
To start this, I made few simple goals for the phase one:
1 A trace ID for linking user's activities
e.g clicking a search button, viewing one of the recalled results...etc
2 track if a user has any interest about the results he just got
e.g if a user interact with any recalled results(watch a video in my case), this search would be determined as a 'postive search', few additional information would be recorded. In contrast, a 'negative search' would be determined if a user got empty result after his search, or no further interaction after the results retrieved.
3 search score normalization: we need to know how the recalled results match user's search attempt as well as which results of the searches are considered as 'relevant'. Normalising the score of the hit from Elasticsearch might be a solution as far as I know. Here is the current solution (quote from lucene - How to normalize ElasticSearch scores? - Stack Overflow ):
A: a pilot query (preferably with size 1, but rest all attributes same) and it will fetch you the max_score
B: shoot your actual query and use functional_score to normalize the score. Pass the max_score you got as part of the pilot query in params to function_score and use it to normalize every score
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.