# Disabling Elasticsearch Inverse Document Frequency scoring on ES relevance score
**URL:** https://discuss.elastic.co/t/disabling-elasticsearch-inverse-document-frequency-scoring-on-es-relevance-score/75145
**Category:** Elasticsearch
**Created:** [February 15, 2017, 7:08am UTC](https://discuss.elastic.co/t/disabling-elasticsearch-inverse-document-frequency-scoring-on-es-relevance-score/75145 "2017-02-15T07:08:21Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@patlola](https://discuss.elastic.co/u/patlola)
#### Post date: [February 15, 2017, 7:08am UTC](https://discuss.elastic.co/t/disabling-elasticsearch-inverse-document-frequency-scoring-on-es-relevance-score/75145/1 "2017-02-15T07:08:21Z")
Am getting irrelevant results when doing a simple match query on the documents.
My documents have a lot of duplicates words but all words are important like.
Documents are:
Doc\_1) Nestle
Doc\_2) Nestle Kitkat
DOc\_3)nestle chocolate
doc\_4)nestle candy
doc\_5) nestle yoghurt
doc\_6) nestle fruit
--hundreds of same kinda documents.
now on doing a simple match query on **\> nestle** it's not scoring Doc\_1 first because it's repeated number of times and it's not relevant because of IDF.
I have tried disabling norms using **norms:{"enabled": false}, index\_options:"docs"**
in the field mapping but still am not getting relevant results.
```
{
"query": {
"bool": {
"disable_coord": true,
"should": [
{
"match": {
"choclate.name": {
"query": "Nestle",
"operator": "and"
}
}
},
{
"match": {
"choclate.whitespace": {
"query": "Nestle",
"operator": "and"
}
}
}
]
}
},
"from": 0,
"size": 1
}
```
do i need to use custom score fucntion ?
if i use custom score function then i won't get other relevance scoring features like field length and normalization etc etc.
---
### Author:  [@jpountz](https://discuss.elastic.co/u/jpountz)
#### Post date: [February 15, 2017, 9:00am UTC](https://discuss.elastic.co/t/disabling-elasticsearch-inverse-document-frequency-scoring-on-es-relevance-score/75145/2 "2017-02-15T09:00:09Z")
Unfortunately this is the kind of requirement that would require to plug in a custom similarity. This issue might do what you want: [https://github.com/elastic/elasticsearch/issues/6731](https://github.com/elastic/elasticsearch/issues/6731). It proposes to add a new similarity that does not take term frequency or document frequency into account, just the number of matching clauses, and document length if norms are enabled. You can upvote it if you think that would address your requirements.
---
### Author:  [@patlola](https://discuss.elastic.co/u/patlola)
#### Post date: [February 15, 2017, 9:04am UTC](https://discuss.elastic.co/t/disabling-elasticsearch-inverse-document-frequency-scoring-on-es-relevance-score/75145/3 "2017-02-15T09:04:43Z")
Thanks for your reply @jpountz
instead of IDF i would like to have a DF only.
is there any possibility to do that ?
---
### Author:  [@Sumit\_Gupta](https://discuss.elastic.co/u/Sumit_Gupta)
#### Post date: [February 15, 2017, 9:05am UTC](https://discuss.elastic.co/t/disabling-elasticsearch-inverse-document-frequency-scoring-on-es-relevance-score/75145/4 "2017-02-15T09:05:51Z")
@jpountz Look like both of these requirements are same
> [@Calculating with Document Frequency, not Inverse Document Frequency](https://discuss.elastic.co/t/calculating-with-document-frequency-not-inverse-document-frequency/12479):
>
> Hello, I have gist [https://gist.github.com/anonymous/5813541](https://gist.github.com/anonymous/5813541) where I search for 'small'. But result where 'small' is just once is higher then the result where 'small' is twice. I want the results where the more frequent the 'small' is, the higher in the results. So, I want DF, not IDF. Is there any way to disable IDF or override idf() in Similarity class? Or how can I solve the problem? Thanks in advance. -- You received this message because you are subscribed to the Google Groups "ela…
---
### Author:  [@jpountz](https://discuss.elastic.co/u/jpountz)
#### Post date: [February 15, 2017, 9:14am UTC](https://discuss.elastic.co/t/disabling-elasticsearch-inverse-document-frequency-scoring-on-es-relevance-score/75145/5 "2017-02-15T09:14:33Z")
Then you would need a custom similarity indeed. Similarities in Lucene are not designed for being extended, but it should be fairly easy to copy-paste an existing impl (typically bm25) and adapt it to your needs.
---
### Author:  [@softwaredoug](https://discuss.elastic.co/u/softwaredoug)
#### Post date: [February 15, 2017, 10:27pm UTC](https://discuss.elastic.co/t/disabling-elasticsearch-inverse-document-frequency-scoring-on-es-relevance-score/75145/6 "2017-02-15T22:27:06Z")
One solution can be to set k to 0 in BM25, which I believe gives you just the BM25 IDF as an implementation when I look at the formula. More [here](http://opensourceconnections.com/blog/2015/10/16/bm25-the-next-generation-of-lucene-relevation/).
---
### Author:  [@sdauletau](https://discuss.elastic.co/u/sdauletau)
#### Post date: [February 16, 2017, 1:25am UTC](https://discuss.elastic.co/t/disabling-elasticsearch-inverse-document-frequency-scoring-on-es-relevance-score/75145/7 "2017-02-16T01:25:09Z")
I have a plugin that ignores tf-idf.
> **[sdauletau/elasticsearch-simple-similarity](https://github.com/sdauletau/elasticsearch-simple-similarity)**
>
> elasticsearch-simple-similarity - Elasticsearch "ignore tf-idf" plugin
You can use it as an example and you can update tf and or idf methods to return term or document frequency.
>
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 16, 2017, 1:25am UTC](https://discuss.elastic.co/t/disabling-elasticsearch-inverse-document-frequency-scoring-on-es-relevance-score/75145/8 "2017-03-16T01:25:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.