# Best option for scoring documents based on custom relevancy score
**URL:** https://discuss.elastic.co/t/best-option-for-scoring-documents-based-on-custom-relevancy-score/19493
**Category:** Elasticsearch
**Created:** [August 27, 2014, 12:52pm UTC](https://discuss.elastic.co/t/best-option-for-scoring-documents-based-on-custom-relevancy-score/19493 "2014-08-27T12:52:46Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@hespoddi](https://discuss.elastic.co/u/hespoddi)
#### Post date: [August 27, 2014, 12:52pm UTC](https://discuss.elastic.co/t/best-option-for-scoring-documents-based-on-custom-relevancy-score/19493/1 "2014-08-27T12:52:46Z")
Hi all,
I have a set of extracted terms, with associated relevancy scores and other
metadata, from each document. I'd like to scale \_score by the relevancy of
the matched terms. It seems to me there are at least two approaches for
solving this problem:
1. Nested Document:
In this case, my mapping would look like:
{
"contentDocument": {
"properties": {
"content": {
"type": "string"
},
"terms": {
"type": "nested",
"fields": {
"text": {
"type": "string",
},
"relevance": {
"type": "float"
}
}
}
}
}
}
Then I could query using:
{
"query": {
"nested": {
"score\_mode": "max",
"path": "terms",
"query": {
"function\_score": {
"boost\_mode": "replace",
"score\_mode": "multiply",
"query": {
"match": {
"terms.text": ""
}
},
"functions": [
{
"field\_value\_factor": {
"field": "terms.relevance"
}
}
]
}
}
}
}
}
This seems to work as expected on the small prototype I've built.
1. Parent/Child Documents:
In this case, my mapping would look like:
{
"contentDocument": {
"properties": {
"content": {
"type": "string"
}
}
}
}
{
"termDocument": {
"\_parent": {
"type": "contentDocument"
},
"properties": {
"text": {
"type": "string"
},
"relevance": {
"type": "float"
}
}
}
}
Then I could query using:
{
"query": {
"has\_child": {
"type": "termDocument",
"score\_mode": "max",
"query": {
"function\_score": {
"boost\_mode": "replace",
"score\_mode": "multiply",
"query": {
"match": {
"text": ""
}
},
"functions": [
{
"field\_value\_factor": {
"field": "termDocument.relevance"
}
}
]
}
}
}
}
}
This also seems to work in the prototype.
So, both options seem to work, which is great! However, I'm not sure if
there are any performance (or other) concerns with approaches? We will have
millions of documents (and associated terms), so we need our solution to
scale well. It seems to me that the nested approach is conceptually more
straightforward, so I'm leaning in that directly, but wanted to get input
for larger ES community.
Please let me know if there is any other options that might work better!
I've also considered using payloads:
[https://groups.google.com/forum/#!searchin/elasticsearch/Scott$20Decker|sort:date/elasticsearch/gEcBVhSynnY/4N1XD5NyseMJ](https://groups.google.com/forum/#!searchin/elasticsearch/Scott%2420Decker%7Csort:date/elasticsearch/gEcBVhSynnY/4N1XD5NyseMJ)
However, I'm not sure that will work for us as there is metadata, other
than relevancy, I'd like to store about each term.
Thank you!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/92296554-97e0-4755-a648-72224c58fea4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/92296554-97e0-4755-a648-72224c58fea4%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [August 27, 2014, 2:24pm UTC](https://discuss.elastic.co/t/best-option-for-scoring-documents-based-on-custom-relevancy-score/19493/2 "2014-08-27T14:24:06Z")
Payloads work better when you need to increase the relevancy at the term
level. Using a field that contains the relevancy boost is applied at the
document level, so it depends on how specific you need your boosting to be.
Boosting at the document level will be more efficient since you are looking
up only one value.
Scoring with a function\_score will always have some impact, but it should
be relatively minor when using field\_value\_factor, which is probably the
best document level boosting mechanism you could use.
Cheers,
Ivan
On Wed, Aug 27, 2014 at 5:52 AM, hespoddi [chris@publishthis.com](mailto:chris@publishthis.com) wrote:
> Hi all,
>
> I have a set of extracted terms, with associated relevancy scores and
> other metadata, from each document. I'd like to scale \_score by the
> relevancy of the matched terms. It seems to me there are at least two
> approaches for solving this problem:
>
> 1. Nested Document:
>
> In this case, my mapping would look like:
>
> {
> "contentDocument": {
> "properties": {
> "content": {
> "type": "string"
> },
> "terms": {
> "type": "nested",
> "fields": {
> "text": {
> "type": "string",
> },
> "relevance": {
> "type": "float"
> }
> }
> }
> }
> }
> }
>
> Then I could query using:
>
> {
> "query": {
> "nested": {
> "score\_mode": "max",
> "path": "terms",
> "query": {
> "function\_score": {
> "boost\_mode": "replace",
> "score\_mode": "multiply",
> "query": {
> "match": {
> "terms.text": ""
> }
> },
> "functions": [
> {
> "field\_value\_factor": {
> "field": "terms.relevance"
> }
> }
> ]
> }
> }
> }
> }
> }
>
> This seems to work as expected on the small prototype I've built.
>
> 1. Parent/Child Documents:
>
> In this case, my mapping would look like:
>
> {
> "contentDocument": {
> "properties": {
> "content": {
> "type": "string"
> }
> }
> }
> }
> {
> "termDocument": {
> "\_parent": {
> "type": "contentDocument"
> },
> "properties": {
> "text": {
> "type": "string"
> },
> "relevance": {
> "type": "float"
> }
> }
> }
> }
>
> Then I could query using:
>
> {
> "query": {
> "has\_child": {
> "type": "termDocument",
> "score\_mode": "max",
> "query": {
> "function\_score": {
> "boost\_mode": "replace",
> "score\_mode": "multiply",
> "query": {
> "match": {
> "text": ""
> }
> },
> "functions": [
> {
> "field\_value\_factor": {
> "field": "termDocument.relevance"
> }
> }
> ]
> }
> }
> }
> }
> }
>
> This also seems to work in the prototype.
>
> So, both options seem to work, which is great! However, I'm not sure if
> there are any performance (or other) concerns with approaches? We will have
> millions of documents (and associated terms), so we need our solution to
> scale well. It seems to me that the nested approach is conceptually more
> straightforward, so I'm leaning in that directly, but wanted to get input
> for larger ES community.
>
> Please let me know if there is any other options that might work better!
> I've also considered using payloads:
>
> [Redirecting to Google Groups](https://groups.google.com/forum/#!searchin/elasticsearch/Scott$20Decker%7Csort:date/elasticsearch/gEcBVhSynnY/4N1XD5NyseMJ)
>
> However, I'm not sure that will work for us as there is metadata, other
> than relevancy, I'd like to store about each term.
>
> Thank you!
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/92296554-97e0-4755-a648-72224c58fea4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/92296554-97e0-4755-a648-72224c58fea4%40googlegroups.com)
> [https://groups.google.com/d/msgid/elasticsearch/92296554-97e0-4755-a648-72224c58fea4%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/92296554-97e0-4755-a648-72224c58fea4%40googlegroups.com?utm_medium=email&utm_source=footer)
> .
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAouQK1FNny57LJ5uzLnVbwQ2PtmkeNhOp\_g65xgxFpYQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAouQK1FNny57LJ5uzLnVbwQ2PtmkeNhOp_g65xgxFpYQ%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@hespoddi](https://discuss.elastic.co/u/hespoddi)
#### Post date: [August 28, 2014, 3:39pm UTC](https://discuss.elastic.co/t/best-option-for-scoring-documents-based-on-custom-relevancy-score/19493/3 "2014-08-28T15:39:03Z")
Okay... Thanks! I think we're going to go with the nested approach then.
On Wednesday, August 27, 2014 10:24:16 AM UTC-4, Ivan Brusic wrote:
> Payloads work better when you need to increase the relevancy at the term
> level. Using a field that contains the relevancy boost is applied at the
> document level, so it depends on how specific you need your boosting to be.
> Boosting at the document level will be more efficient since you are looking
> up only one value.
>
> Scoring with a function\_score will always have some impact, but it should
> be relatively minor when using field\_value\_factor, which is probably the
> best document level boosting mechanism you could use.
>
> Cheers,
>
> Ivan
>
> On Wed, Aug 27, 2014 at 5:52 AM, hespoddi \<[ch...@publishthis.com](mailto:ch...@publishthis.com)
> \\> wrote:
>
> > Hi all,
> >
> > I have a set of extracted terms, with associated relevancy scores and
> > other metadata, from each document. I'd like to scale \_score by the
> > relevancy of the matched terms. It seems to me there are at least two
> > approaches for solving this problem:
> >
> > 1. Nested Document:
> >
> > In this case, my mapping would look like:
> >
> > {
> > "contentDocument": {
> > "properties": {
> > "content": {
> > "type": "string"
> > },
> > "terms": {
> > "type": "nested",
> > "fields": {
> > "text": {
> > "type": "string",
> > },
> > "relevance": {
> > "type": "float"
> > }
> > }
> > }
> > }
> > }
> > }
> >
> > Then I could query using:
> >
> > {
> > "query": {
> > "nested": {
> > "score\_mode": "max",
> > "path": "terms",
> > "query": {
> > "function\_score": {
> > "boost\_mode": "replace",
> > "score\_mode": "multiply",
> > "query": {
> > "match": {
> > "terms.text": ""
> > }
> > },
> > "functions": [
> > {
> > "field\_value\_factor": {
> > "field": "terms.relevance"
> > }
> > }
> > ]
> > }
> > }
> > }
> > }
> > }
> >
> > This seems to work as expected on the small prototype I've built.
> >
> > 1. Parent/Child Documents:
> >
> > In this case, my mapping would look like:
> >
> > {
> > "contentDocument": {
> > "properties": {
> > "content": {
> > "type": "string"
> > }
> > }
> > }
> > }
> > {
> > "termDocument": {
> > "\_parent": {
> > "type": "contentDocument"
> > },
> > "properties": {
> > "text": {
> > "type": "string"
> > },
> > "relevance": {
> > "type": "float"
> > }
> > }
> > }
> > }
> >
> > Then I could query using:
> >
> > {
> > "query": {
> > "has\_child": {
> > "type": "termDocument",
> > "score\_mode": "max",
> > "query": {
> > "function\_score": {
> > "boost\_mode": "replace",
> > "score\_mode": "multiply",
> > "query": {
> > "match": {
> > "text": ""
> > }
> > },
> > "functions": [
> > {
> > "field\_value\_factor": {
> > "field": "termDocument.relevance"
> > }
> > }
> > ]
> > }
> > }
> > }
> > }
> > }
> >
> > This also seems to work in the prototype.
> >
> > So, both options seem to work, which is great! However, I'm not sure if
> > there are any performance (or other) concerns with approaches? We will have
> > millions of documents (and associated terms), so we need our solution to
> > scale well. It seems to me that the nested approach is conceptually more
> > straightforward, so I'm leaning in that directly, but wanted to get input
> > for larger ES community.
> >
> > Please let me know if there is any other options that might work better!
> > I've also considered using payloads:
> >
> > [Redirecting to Google Groups](https://groups.google.com/forum/#!searchin/elasticsearch/Scott$20Decker%7Csort:date/elasticsearch/gEcBVhSynnY/4N1XD5NyseMJ)
> >
> > However, I'm not sure that will work for us as there is metadata, other
> > than relevancy, I'd like to store about each term.
> >
> > Thank you!
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/92296554-97e0-4755-a648-72224c58fea4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/92296554-97e0-4755-a648-72224c58fea4%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/92296554-97e0-4755-a648-72224c58fea4%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/92296554-97e0-4755-a648-72224c58fea4%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/62b4a2c9-518a-4815-81d7-1b60036b48a0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/62b4a2c9-518a-4815-81d7-1b60036b48a0%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:05am UTC](https://discuss.elastic.co/t/best-option-for-scoring-documents-based-on-custom-relevancy-score/19493/4 "2017-07-06T01:05:47Z")