# Custom relevance scoring by term frequency averages
**URL:** https://discuss.elastic.co/t/custom-relevance-scoring-by-term-frequency-averages/17708
**Category:** Elasticsearch
**Created:** [May 24, 2014, 2:22pm UTC](https://discuss.elastic.co/t/custom-relevance-scoring-by-term-frequency-averages/17708 "2014-05-24T14:22:30Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@Chris\_H\_3](https://discuss.elastic.co/u/Chris_H_3)
#### Post date: [May 24, 2014, 2:22pm UTC](https://discuss.elastic.co/t/custom-relevance-scoring-by-term-frequency-averages/17708/1 "2014-05-24T14:22:30Z")
Hi,
I want to calculate relevance scores in a different way from the default
TFIDF scoring in ES.
In particular I want to calculate it simply as:
AVG( tf(term) / ttf(term) )
where the average is over all matching terms in the query.
For example, suppose I have the following documents:
PUT /documents/document/1
{
"content": "test document test"
}
PUT /documents/document/2
{
"content": "another test document"
}
I want scores for the query below for "test document" to be calculated as:
GET /documents/\_search
{
"query": {"match" : {"content": "test document"}}
}
Doc 1:
AVG( tf(test)/ttf(test), tf(document)/ttf(document) ) = AVG(2/3, 1/2) =
_7/12_
Doc 2:
AVG ( tf(test)/ttf(test), tf(document)/ttf(document) ) = AVG(1/3, 1/2) =
_5/12_
Is there any way I can achieve this in Elasticsearch?
Later I may want to weight the averages by IDF or document length, but at
the moment I just want to do the above.
Any help greatly appreciated.
Thanks
Chris
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5147a6e9-9a9e-4844-b153-50696a2ecc06%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5147a6e9-9a9e-4844-b153-50696a2ecc06%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [May 24, 2014, 2:48pm UTC](https://discuss.elastic.co/t/custom-relevance-scoring-by-term-frequency-averages/17708/2 "2014-05-24T14:48:31Z")
You have a couple of options. The first is writing your own similarity
class which (subclasses TFIDF or
[DefaultSimilarity (Lucene 4.8.1 API)](http://lucene.apache.org/core/4_8_1/core/org/apache/lucene/search/similarities/DefaultSimilarity.html))
and you would override the various methods. I find this option easier,
however I do not think you will be able to access the distributed term
frequencies, so it would work in cases with only one shard or if you do not
mind have potential slight inconsistencies. The more data you have, the
more the non-distributed frequencies even out.
The other option would be to use function scoring. There are some text
scoring examples on the site:
> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.
If you want to provide a native Java solution, Britta (who wrote much of
the funtion scoring code) contributed examples to Igor's native script
example repo: [GitHub - imotov/elasticsearch-native-script-example: Example of Now Deprecated Native Script Plugin for Elasticsearch](https://github.com/imotov/elasticsearch-native-script-example)
Of interest:
[https://github.com/imotov/elasticsearch-native-script-example/blob/master/src/main/java/org/elasticsearch/examples/nativescript/script/TFIDFScoreScript.java](https://github.com/imotov/elasticsearch-native-script-example/blob/master/src/main/java/org/elasticsearch/examples/nativescript/script/TFIDFScoreScript.java)
[https://github.com/imotov/elasticsearch-native-script-example/blob/master/src/main/java/org/elasticsearch/examples/nativescript/script/CosineSimilarityScoreScript.java](https://github.com/imotov/elasticsearch-native-script-example/blob/master/src/main/java/org/elasticsearch/examples/nativescript/script/CosineSimilarityScoreScript.java)
Cheers,
Ivan
On Sat, May 24, 2014 at 7:22 AM, Chris H [c.harper80@gmail.com](mailto:c.harper80@gmail.com) wrote:
> Hi,
>
> I want to calculate relevance scores in a different way from the default
> TFIDF scoring in ES.
> In particular I want to calculate it simply as:
>
> AVG( tf(term) / ttf(term) )
>
> where the average is over all matching terms in the query.
>
> For example, suppose I have the following documents:
>
> PUT /documents/document/1
> {
> "content": "test document test"
> }
>
> PUT /documents/document/2
> {
> "content": "another test document"
> }
>
> I want scores for the query below for "test document" to be calculated as:
>
> GET /documents/\_search
> {
> "query": {"match" : {"content": "test document"}}
> }
>
> Doc 1:
> AVG( tf(test)/ttf(test), tf(document)/ttf(document) ) = AVG(2/3, 1/2) =
> _7/12_
>
> Doc 2:
> AVG ( tf(test)/ttf(test), tf(document)/ttf(document) ) = AVG(1/3, 1/2) =
> _5/12_
>
> Is there any way I can achieve this in Elasticsearch?
> Later I may want to weight the averages by IDF or document length, but at
> the moment I just want to do the above.
> Any help greatly appreciated.
>
> Thanks
> Chris
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/5147a6e9-9a9e-4844-b153-50696a2ecc06%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5147a6e9-9a9e-4844-b153-50696a2ecc06%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/5147a6e9-9a9e-4844-b153-50696a2ecc06%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/5147a6e9-9a9e-4844-b153-50696a2ecc06%40googlegroups.com?utm_medium=email&utm_source=footer)
> .
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBZyuTctggTL-1wwtqv7m%3DH1%2Bkgf\_kPTYqL9wrvKK%2B4wg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBZyuTctggTL-1wwtqv7m%3DH1%2Bkgf_kPTYqL9wrvKK%2B4wg%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:27am UTC](https://discuss.elastic.co/t/custom-relevance-scoring-by-term-frequency-averages/17708/3 "2017-07-06T01:27:12Z")