# Combing relevancy and trending scores
**URL:** https://discuss.elastic.co/t/combing-relevancy-and-trending-scores/21213
**Category:** Elasticsearch
**Created:** [December 11, 2014, 1:58pm UTC](https://discuss.elastic.co/t/combing-relevancy-and-trending-scores/21213 "2014-12-11T13:58:54Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@hespoddi](https://discuss.elastic.co/u/hespoddi)
#### Post date: [December 11, 2014, 1:58pm UTC](https://discuss.elastic.co/t/combing-relevancy-and-trending-scores/21213/1 "2014-12-11T13:58:54Z")
Hi all,
We'd like to combine the query score with our own custom trending score for
a given document. Currently, our query looks like:
{
"query": {
"filtered": {
"filter": {
"and": [
{
"range": {
"trendingScore": {
"gt": 0.0
}
}
}
]
},
"query": {
"function\_score": {
"functions": [
{
"field\_value\_factor": {
"field": "trendingScore"
}
}
],
"query": {
"bool": {
"minimum\_should\_match": 1,
"should": [
{
"match\_phrase": {
"title": {
"boost": 1.5,
"query": ""
}
}
},
{
"match\_phrase": {
"content": {
"boost": 1.0,
"query": ""
}
}
}
]
}
}
}
}
}
}
}
So, this query seems to do want we want: it multiples the query score for
document with our custom trending score (stored on the "trendingScore"
field). The problem is that the trending score, in many cases, overwhelms
the query score. Thus, documents with very low relevancy, but very high
trending, are at the top of our results. Ideally, we'd filter the query to
only return the top N percentage of documents that matched, but I don't
think that's possible. We've looked at the min\_score parameter for queries
as well, but I don't know what a "good" value would be for this.
Does anyone have any ideas on the best way to solve this problem? Thank you
ahead of time!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@vineeth\_mohan\_2](https://discuss.elastic.co/u/vineeth_mohan_2)
#### Post date: [December 22, 2014, 4:01am UTC](https://discuss.elastic.co/t/combing-relevancy-and-trending-scores/21213/2 "2014-12-22T04:01:00Z")
Hello ,
My advice would be to use the script function type. Inside it you can
access \_score which is the score given by the query and the value of the
field. Mix them together in whatever logic you want.
Thanks
Vineeth
On Thu, Dec 11, 2014 at 7:28 PM, hespoddi [chris@publishthis.com](mailto:chris@publishthis.com) wrote:
> Hi all,
>
> We'd like to combine the query score with our own custom trending score
> for a given document. Currently, our query looks like:
>
> {
> "query": {
> "filtered": {
> "filter": {
> "and": [
> {
> "range": {
> "trendingScore": {
> "gt": 0.0
> }
> }
> }
> ]
> },
> "query": {
> "function\_score": {
> "functions": [
> {
> "field\_value\_factor": {
> "field": "trendingScore"
> }
> }
> ],
> "query": {
> "bool": {
> "minimum\_should\_match": 1,
> "should": [
> {
> "match\_phrase": {
> "title": {
> "boost": 1.5,
> "query": ""
> }
> }
> },
> {
> "match\_phrase": {
> "content": {
> "boost": 1.0,
> "query": ""
> }
> }
> }
> ]
> }
> }
> }
> }
> }
> }
> }
>
> So, this query seems to do want we want: it multiples the query score for
> document with our custom trending score (stored on the "trendingScore"
> field). The problem is that the trending score, in many cases, overwhelms
> the query score. Thus, documents with very low relevancy, but very high
> trending, are at the top of our results. Ideally, we'd filter the query to
> only return the top N percentage of documents that matched, but I don't
> think that's possible. We've looked at the min\_score parameter for queries
> as well, but I don't know what a "good" value would be for this.
>
> Does anyone have any ideas on the best way to solve this problem? Thank
> you ahead of time!
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com)
> [https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com?utm_medium=email&utm_source=footer)
> .
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGdPd5nkqQhECx%3DhqPtdApGLU1T06roOabeds%2Btr2yrB6pmxKg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGdPd5nkqQhECx%3DhqPtdApGLU1T06roOabeds%2Btr2yrB6pmxKg%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@Scott\_Decker](https://discuss.elastic.co/u/Scott_Decker)
#### Post date: [December 22, 2014, 5:00pm UTC](https://discuss.elastic.co/t/combing-relevancy-and-trending-scores/21213/3 "2014-12-22T17:00:58Z")
If you did the script score path, don't you run into performance issues? I
would think, running say thousands of queries like that would probably not
be performant.
unless I am missing something.
On Sunday, December 21, 2014 8:01:06 PM UTC-8, vineeth mohan wrote:
> Hello ,
>
> My advice would be to use the script function type. Inside it you can
> access \_score which is the score given by the query and the value of the
> field. Mix them together in whatever logic you want.
>
> Thanks
> Vineeth
>
> On Thu, Dec 11, 2014 at 7:28 PM, hespoddi \<[ch...@publishthis.com](mailto:ch...@publishthis.com)
> \\> wrote:
>
> > Hi all,
> >
> > We'd like to combine the query score with our own custom trending score
> > for a given document. Currently, our query looks like:
> >
> > {
> > "query": {
> > "filtered": {
> > "filter": {
> > "and": [
> > {
> > "range": {
> > "trendingScore": {
> > "gt": 0.0
> > }
> > }
> > }
> > ]
> > },
> > "query": {
> > "function\_score": {
> > "functions": [
> > {
> > "field\_value\_factor": {
> > "field": "trendingScore"
> > }
> > }
> > ],
> > "query": {
> > "bool": {
> > "minimum\_should\_match": 1,
> > "should": [
> > {
> > "match\_phrase": {
> > "title": {
> > "boost": 1.5,
> > "query": ""
> > }
> > }
> > },
> > {
> > "match\_phrase": {
> > "content": {
> > "boost": 1.0,
> > "query": ""
> > }
> > }
> > }
> > ]
> > }
> > }
> > }
> > }
> > }
> > }
> > }
> >
> > So, this query seems to do want we want: it multiples the query score for
> > document with our custom trending score (stored on the "trendingScore"
> > field). The problem is that the trending score, in many cases, overwhelms
> > the query score. Thus, documents with very low relevancy, but very high
> > trending, are at the top of our results. Ideally, we'd filter the query to
> > only return the top N percentage of documents that matched, but I don't
> > think that's possible. We've looked at the min\_score parameter for queries
> > as well, but I don't know what a "good" value would be for this.
> >
> > Does anyone have any ideas on the best way to solve this problem? Thank
> > you ahead of time!
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@vineeth\_mohan\_2](https://discuss.elastic.co/u/vineeth_mohan_2)
#### Post date: [December 22, 2014, 6:34pm UTC](https://discuss.elastic.co/t/combing-relevancy-and-trending-scores/21213/4 "2014-12-22T18:34:02Z")
Hi ,
I dont see why it should run into performance issue.
In anyway you do it , the \_score and score due to a field have to be
computed/loaded.
If you precompile your script by placing it in config directory , that
should be good enough.
Also feel free to write the same in Java code and attach that as script.
Thanks
Vineeth
On Mon, Dec 22, 2014 at 10:30 PM, Scott Decker [scott@publishthis.com](mailto:scott@publishthis.com)
wrote:
> If you did the script score path, don't you run into performance issues?
> I would think, running say thousands of queries like that would probably
> not be performant.
> unless I am missing something.
>
> On Sunday, December 21, 2014 8:01:06 PM UTC-8, vineeth mohan wrote:
>
> > Hello ,
> >
> > My advice would be to use the script function type. Inside it you can
> > access \_score which is the score given by the query and the value of the
> > field. Mix them together in whatever logic you want.
> >
> > Thanks
> > Vineeth
> >
> > On Thu, Dec 11, 2014 at 7:28 PM, hespoddi [ch...@publishthis.com](mailto:ch...@publishthis.com) wrote:
> >
> > > Hi all,
> > >
> > > We'd like to combine the query score with our own custom trending score
> > > for a given document. Currently, our query looks like:
> > >
> > > {
> > > "query": {
> > > "filtered": {
> > > "filter": {
> > > "and": [
> > > {
> > > "range": {
> > > "trendingScore": {
> > > "gt": 0.0
> > > }
> > > }
> > > }
> > > ]
> > > },
> > > "query": {
> > > "function\_score": {
> > > "functions": [
> > > {
> > > "field\_value\_factor": {
> > > "field": "trendingScore"
> > > }
> > > }
> > > ],
> > > "query": {
> > > "bool": {
> > > "minimum\_should\_match": 1,
> > > "should": [
> > > {
> > > "match\_phrase": {
> > > "title": {
> > > "boost": 1.5,
> > > "query": ""
> > > }
> > > }
> > > },
> > > {
> > > "match\_phrase": {
> > > "content": {
> > > "boost": 1.0,
> > > "query": ""
> > > }
> > > }
> > > }
> > > ]
> > > }
> > > }
> > > }
> > > }
> > > }
> > > }
> > > }
> > >
> > > So, this query seems to do want we want: it multiples the query score
> > > for document with our custom trending score (stored on the "trendingScore"
> > > field). The problem is that the trending score, in many cases, overwhelms
> > > the query score. Thus, documents with very low relevancy, but very high
> > > trending, are at the top of our results. Ideally, we'd filter the query to
> > > only return the top N percentage of documents that matched, but I don't
> > > think that's possible. We've looked at the min\_score parameter for queries
> > > as well, but I don't know what a "good" value would be for this.
> > >
> > > Does anyone have any ideas on the best way to solve this problem? Thank
> > > you ahead of time!
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "elasticsearch" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%
> > > [40googlegroups.com](http://40googlegroups.com)
> > > [https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > .
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
>
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGdPd5kMXpM6r%3DkS3Ef7qJB8nmqVZYcp6KGHoOWTJSRa17ivBA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGdPd5kMXpM6r%3DkS3Ef7qJB8nmqVZYcp6KGHoOWTJSRa17ivBA%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@hespoddi](https://discuss.elastic.co/u/hespoddi)
#### Post date: [December 22, 2014, 9:16pm UTC](https://discuss.elastic.co/t/combing-relevancy-and-trending-scores/21213/5 "2014-12-22T21:16:58Z")
Hi all,
The problem isn't really the query. The problem is we'd like the limit the
results of the query to just "high" scores before we apply the
function\_score. There is a min\_score parameter we could use:
> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.
But what the min\_score should be will, obviously, vary significantly
depending on the query. Ideally, we'd set the min\_score to some percentile
of the max score for query, but I don't think that's possible:
>
>
> Allow to filter docs based on a minimum score, for example:
>
> \`\`\`
> {
> "min\_scor…e": 0.5,
> "query" : {
> "term" : { "user" : "kimchy" }
> }
> }
> \`\`\`
I was curious if anyone had any other ideas about how to do this (or
something close)?
-Chris
On Monday, December 22, 2014 1:34:08 PM UTC-5, vineeth mohan wrote:
> Hi ,
>
> I dont see why it should run into performance issue.
> In anyway you do it , the \_score and score due to a field have to be
> computed/loaded.
> If you precompile your script by placing it in config directory , that
> should be good enough.
> Also feel free to write the same in Java code and attach that as script.
>
> Thanks
> Vineeth
>
> On Mon, Dec 22, 2014 at 10:30 PM, Scott Decker \<[sc...@publishthis.com](mailto:sc...@publishthis.com)
> \\> wrote:
>
> > If you did the script score path, don't you run into performance issues?
> > I would think, running say thousands of queries like that would probably
> > not be performant.
> > unless I am missing something.
> >
> > On Sunday, December 21, 2014 8:01:06 PM UTC-8, vineeth mohan wrote:
> >
> > > Hello ,
> > >
> > > My advice would be to use the script function type. Inside it you can
> > > access \_score which is the score given by the query and the value of the
> > > field. Mix them together in whatever logic you want.
> > >
> > > Thanks
> > > Vineeth
> > >
> > > On Thu, Dec 11, 2014 at 7:28 PM, hespoddi [ch...@publishthis.com](mailto:ch...@publishthis.com) wrote:
> > >
> > > > Hi all,
> > > >
> > > > We'd like to combine the query score with our own custom trending score
> > > > for a given document. Currently, our query looks like:
> > > >
> > > > {
> > > > "query": {
> > > > "filtered": {
> > > > "filter": {
> > > > "and": [
> > > > {
> > > > "range": {
> > > > "trendingScore": {
> > > > "gt": 0.0
> > > > }
> > > > }
> > > > }
> > > > ]
> > > > },
> > > > "query": {
> > > > "function\_score": {
> > > > "functions": [
> > > > {
> > > > "field\_value\_factor": {
> > > > "field": "trendingScore"
> > > > }
> > > > }
> > > > ],
> > > > "query": {
> > > > "bool": {
> > > > "minimum\_should\_match": 1,
> > > > "should": [
> > > > {
> > > > "match\_phrase": {
> > > > "title": {
> > > > "boost": 1.5,
> > > > "query": ""
> > > > }
> > > > }
> > > > },
> > > > {
> > > > "match\_phrase": {
> > > > "content": {
> > > > "boost": 1.0,
> > > > "query": ""
> > > > }
> > > > }
> > > > }
> > > > ]
> > > > }
> > > > }
> > > > }
> > > > }
> > > > }
> > > > }
> > > > }
> > > >
> > > > So, this query seems to do want we want: it multiples the query score
> > > > for document with our custom trending score (stored on the "trendingScore"
> > > > field). The problem is that the trending score, in many cases, overwhelms
> > > > the query score. Thus, documents with very low relevancy, but very high
> > > > trending, are at the top of our results. Ideally, we'd filter the query to
> > > > only return the top N percentage of documents that matched, but I don't
> > > > think that's possible. We've looked at the min\_score parameter for queries
> > > > as well, but I don't know what a "good" value would be for this.
> > > >
> > > > Does anyone have any ideas on the best way to solve this problem? Thank
> > > > you ahead of time!
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google
> > > > Groups "elasticsearch" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%
> > > > [40googlegroups.com](http://40googlegroups.com)
> > > > [https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > .
> > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > >
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "elasticsearch" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an
> > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > > To view this discussion on the web visit
> > > [https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com)
> > > [https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > .
> >
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/304f009b-ddc3-4826-8686-cc653c81d946%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/304f009b-ddc3-4826-8686-cc653c81d946%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@vineeth\_mohan\_2](https://discuss.elastic.co/u/vineeth_mohan_2)
#### Post date: [December 23, 2014, 4:50am UTC](https://discuss.elastic.co/t/combing-relevancy-and-trending-scores/21213/6 "2014-12-23T04:50:17Z")
Hi ,
Maybe you are looking for this -
> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.
Thanks
Vineeth
On Tue, Dec 23, 2014 at 2:46 AM, hespoddi [chris@publishthis.com](mailto:chris@publishthis.com) wrote:
> Hi all,
>
> The problem isn't really the query. The problem is we'd like the limit the
> results of the query to just "high" scores before we apply the
> function\_score. There is a min\_score parameter we could use:
>
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-min-score.html)
>
> But what the min\_score should be will, obviously, vary significantly
> depending on the query. Ideally, we'd set the min\_score to some percentile
> of the max score for query, but I don't think that's possible:
>
> [Search: Allow to filter out docs based on a minimum score · Issue #719 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/719)
>
> I was curious if anyone had any other ideas about how to do this (or
> something close)?
>
> -Chris
>
> On Monday, December 22, 2014 1:34:08 PM UTC-5, vineeth mohan wrote:
>
> > Hi ,
> >
> > I dont see why it should run into performance issue.
> > In anyway you do it , the \_score and score due to a field have to be
> > computed/loaded.
> > If you precompile your script by placing it in config directory , that
> > should be good enough.
> > Also feel free to write the same in Java code and attach that as script.
> >
> > Thanks
> > Vineeth
> >
> > On Mon, Dec 22, 2014 at 10:30 PM, Scott Decker [sc...@publishthis.com](mailto:sc...@publishthis.com)
> > wrote:
> >
> > > If you did the script score path, don't you run into performance
> > > issues? I would think, running say thousands of queries like that would
> > > probably not be performant.
> > > unless I am missing something.
> > >
> > > On Sunday, December 21, 2014 8:01:06 PM UTC-8, vineeth mohan wrote:
> > >
> > > > Hello ,
> > > >
> > > > My advice would be to use the script function type. Inside it you can
> > > > access \_score which is the score given by the query and the value of the
> > > > field. Mix them together in whatever logic you want.
> > > >
> > > > Thanks
> > > > Vineeth
> > > >
> > > > On Thu, Dec 11, 2014 at 7:28 PM, hespoddi [ch...@publishthis.com](mailto:ch...@publishthis.com)
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > We'd like to combine the query score with our own custom trending
> > > > > score for a given document. Currently, our query looks like:
> > > > >
> > > > > {
> > > > > "query": {
> > > > > "filtered": {
> > > > > "filter": {
> > > > > "and": [
> > > > > {
> > > > > "range": {
> > > > > "trendingScore": {
> > > > > "gt": 0.0
> > > > > }
> > > > > }
> > > > > }
> > > > > ]
> > > > > },
> > > > > "query": {
> > > > > "function\_score": {
> > > > > "functions": [
> > > > > {
> > > > > "field\_value\_factor": {
> > > > > "field": "trendingScore"
> > > > > }
> > > > > }
> > > > > ],
> > > > > "query": {
> > > > > "bool": {
> > > > > "minimum\_should\_match": 1,
> > > > > "should": [
> > > > > {
> > > > > "match\_phrase": {
> > > > > "title": {
> > > > > "boost": 1.5,
> > > > > "query": ""
> > > > > }
> > > > > }
> > > > > },
> > > > > {
> > > > > "match\_phrase": {
> > > > > "content": {
> > > > > "boost": 1.0,
> > > > > "query": ""
> > > > > }
> > > > > }
> > > > > }
> > > > > ]
> > > > > }
> > > > > }
> > > > > }
> > > > > }
> > > > > }
> > > > > }
> > > > > }
> > > > >
> > > > > So, this query seems to do want we want: it multiples the query score
> > > > > for document with our custom trending score (stored on the "trendingScore"
> > > > > field). The problem is that the trending score, in many cases, overwhelms
> > > > > the query score. Thus, documents with very low relevancy, but very high
> > > > > trending, are at the top of our results. Ideally, we'd filter the query to
> > > > > only return the top N percentage of documents that matched, but I don't
> > > > > think that's possible. We've looked at the min\_score parameter for queries
> > > > > as well, but I don't know what a "good" value would be for this.
> > > > >
> > > > > Does anyone have any ideas on the best way to solve this problem?
> > > > > Thank you ahead of time!
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > > > Groups "elasticsearch" group.
> > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40goo
> > > > > [glegroups.com](http://glegroups.com)
> > > > > [https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > > .
> > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google
> > > > Groups "elasticsearch" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%
> > > > [40googlegroups.com](http://40googlegroups.com)
> > > > [https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > .
> > >
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/304f009b-ddc3-4826-8686-cc653c81d946%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/304f009b-ddc3-4826-8686-cc653c81d946%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/304f009b-ddc3-4826-8686-cc653c81d946%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/304f009b-ddc3-4826-8686-cc653c81d946%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
>
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGdPd5m\_vh04Xi0D%2BdXtjL-aNjEGQ42SV92pUS%2BjAdqiijPjkQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGdPd5m_vh04Xi0D%2BdXtjL-aNjEGQ42SV92pUS%2BjAdqiijPjkQ%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@hespoddi](https://discuss.elastic.co/u/hespoddi)
#### Post date: [December 23, 2014, 5:30pm UTC](https://discuss.elastic.co/t/combing-relevancy-and-trending-scores/21213/7 "2014-12-23T17:30:03Z")
We originally thought rescoring would work as well! I actually implemented
it, but low relevancy documents continued to show up at the top of our
results. I didn't understand this at first, but on re-reading the
documentation I saw what the problem was: the rescore is executed on the
shard _before_ the results are returned to the node handling the overall
request. So: if an individual shard only had low relevancy, high trending
documents for a query, then we'd run into the exact same problem as before.
And, in fact, that does seem to happen with enough frequency to be an issue
for us.
At the moment, the only way I see to solve the problem is to do a
post-processing step on the returned documents.
On Monday, December 22, 2014 11:50:24 PM UTC-5, vineeth mohan wrote:
> Hi ,
>
> Maybe you are looking for this -
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-rescore.html)
>
> Thanks
> Vineeth
>
> On Tue, Dec 23, 2014 at 2:46 AM, hespoddi \<[ch...@publishthis.com](mailto:ch...@publishthis.com)
> \\> wrote:
>
> > Hi all,
> >
> > The problem isn't really the query. The problem is we'd like the limit
> > the results of the query to just "high" scores before we apply the
> > function\_score. There is a min\_score parameter we could use:
> >
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-min-score.html)
> >
> > But what the min\_score should be will, obviously, vary significantly
> > depending on the query. Ideally, we'd set the min\_score to some percentile
> > of the max score for query, but I don't think that's possible:
> >
> > [Search: Allow to filter out docs based on a minimum score · Issue #719 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/719)
> >
> > I was curious if anyone had any other ideas about how to do this (or
> > something close)?
> >
> > -Chris
> >
> > On Monday, December 22, 2014 1:34:08 PM UTC-5, vineeth mohan wrote:
> >
> > > Hi ,
> > >
> > > I dont see why it should run into performance issue.
> > > In anyway you do it , the \_score and score due to a field have to be
> > > computed/loaded.
> > > If you precompile your script by placing it in config directory , that
> > > should be good enough.
> > > Also feel free to write the same in Java code and attach that as script.
> > >
> > > Thanks
> > > Vineeth
> > >
> > > On Mon, Dec 22, 2014 at 10:30 PM, Scott Decker [sc...@publishthis.com](mailto:sc...@publishthis.com)
> > > wrote:
> > >
> > > > If you did the script score path, don't you run into performance
> > > > issues? I would think, running say thousands of queries like that would
> > > > probably not be performant.
> > > > unless I am missing something.
> > > >
> > > > On Sunday, December 21, 2014 8:01:06 PM UTC-8, vineeth mohan wrote:
> > > >
> > > > > Hello ,
> > > > >
> > > > > My advice would be to use the script function type. Inside it you can
> > > > > access \_score which is the score given by the query and the value of the
> > > > > field. Mix them together in whatever logic you want.
> > > > >
> > > > > Thanks
> > > > > Vineeth
> > > > >
> > > > > On Thu, Dec 11, 2014 at 7:28 PM, hespoddi [ch...@publishthis.com](mailto:ch...@publishthis.com)
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > We'd like to combine the query score with our own custom trending
> > > > > > score for a given document. Currently, our query looks like:
> > > > > >
> > > > > > {
> > > > > > "query": {
> > > > > > "filtered": {
> > > > > > "filter": {
> > > > > > "and": [
> > > > > > {
> > > > > > "range": {
> > > > > > "trendingScore": {
> > > > > > "gt": 0.0
> > > > > > }
> > > > > > }
> > > > > > }
> > > > > > ]
> > > > > > },
> > > > > > "query": {
> > > > > > "function\_score": {
> > > > > > "functions": [
> > > > > > {
> > > > > > "field\_value\_factor": {
> > > > > > "field": "trendingScore"
> > > > > > }
> > > > > > }
> > > > > > ],
> > > > > > "query": {
> > > > > > "bool": {
> > > > > > "minimum\_should\_match": 1,
> > > > > > "should": [
> > > > > > {
> > > > > > "match\_phrase": {
> > > > > > "title": {
> > > > > > "boost": 1.5,
> > > > > > "query": ""
> > > > > > }
> > > > > > }
> > > > > > },
> > > > > > {
> > > > > > "match\_phrase": {
> > > > > > "content": {
> > > > > > "boost": 1.0,
> > > > > > "query": ""
> > > > > > }
> > > > > > }
> > > > > > }
> > > > > > ]
> > > > > > }
> > > > > > }
> > > > > > }
> > > > > > }
> > > > > > }
> > > > > > }
> > > > > > }
> > > > > >
> > > > > > So, this query seems to do want we want: it multiples the query score
> > > > > > for document with our custom trending score (stored on the "trendingScore"
> > > > > > field). The problem is that the trending score, in many cases, overwhelms
> > > > > > the query score. Thus, documents with very low relevancy, but very high
> > > > > > trending, are at the top of our results. Ideally, we'd filter the query to
> > > > > > only return the top N percentage of documents that matched, but I don't
> > > > > > think that's possible. We've looked at the min\_score parameter for queries
> > > > > > as well, but I don't know what a "good" value would be for this.
> > > > > >
> > > > > > Does anyone have any ideas on the best way to solve this problem?
> > > > > > Thank you ahead of time!
> > > > > >
> > > > > > --
> > > > > > You received this message because you are subscribed to the Google
> > > > > > Groups "elasticsearch" group.
> > > > > > To unsubscribe from this group and stop receiving emails from it,
> > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > > msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40goo
> > > > > > [glegroups.com](http://glegroups.com)
> > > > > > [https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > > > .
> > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > > > Groups "elasticsearch" group.
> > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%
> > > > > [40googlegroups.com](http://40googlegroups.com)
> > > > > [https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > > .
> > > >
> > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > >
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "elasticsearch" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an
> > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > > To view this discussion on the web visit
> > > [https://groups.google.com/d/msgid/elasticsearch/304f009b-ddc3-4826-8686-cc653c81d946%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/304f009b-ddc3-4826-8686-cc653c81d946%40googlegroups.com)
> > > [https://groups.google.com/d/msgid/elasticsearch/304f009b-ddc3-4826-8686-cc653c81d946%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/304f009b-ddc3-4826-8686-cc653c81d946%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > .
> >
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/16ec7395-4d06-4cdf-a1aa-02c6559f6051%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/16ec7395-4d06-4cdf-a1aa-02c6559f6051%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 12:42am UTC](https://discuss.elastic.co/t/combing-relevancy-and-trending-scores/21213/8 "2017-07-06T00:42:10Z")