# Question regarding TF/IDF implementation

**URL:** https://discuss.elastic.co/t/question-regarding-tf-idf-implementation/267933
**Category:** Elasticsearch
**Created:** [March 22, 2021, 3:18am UTC](https://discuss.elastic.co/t/question-regarding-tf-idf-implementation/267933 "2021-03-22T03:18:00Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Yaron\_Golan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaron_golan/32/85877_2.png) [@Yaron\_Golan](https://discuss.elastic.co/u/Yaron_Golan)
#### Post date: [March 22, 2021, 3:18am UTC](https://discuss.elastic.co/t/question-regarding-tf-idf-implementation/267933/1 "2021-03-22T03:18:00Z")

</div>

We have been using an older Elastic version (1.4) for a while and have recently upgraded but wish to continue using the TF/IDF scoring algorithm.

In [Similarity module | Elasticsearch Reference [7.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-similarity.html), there is an example of how to re-implement TF/IDF.

It is:  
"source": "double tf = Math.sqrt(doc.freq); double idf = Math.log((field.docCount+1.0)/(term.docFreq+1.0)) + 1.0; double norm = 1/Math.sqrt(doc.length); return query.boost \* tf \* idf \* norm;"

Why is field.docCount being used instead of simply the number of indexed documents?

---

<div class="post-metadata">

### Author: ![vincenbr](https://avatars.discourse-cdn.com/v4/letter/v/8edcca/32.png) [@vincenbr](https://discuss.elastic.co/u/vincenbr)
#### Post date: [March 22, 2021, 11:25am UTC](https://discuss.elastic.co/t/question-regarding-tf-idf-implementation/267933/2 "2021-03-22T11:25:44Z")

</div>

Hi Yaron, in elasticsearch (even in version 1.4) similarity is per-field. So idf part is based on the field "scope", meaning the count of documents which contain this field.  
In version 7.x , I _think_ (did not test) you can still use tf/idf , even if deprecated, by setting the index-level similarity to "classic", as in [similarity | Elasticsearch Reference [7.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/similarity.html)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 19, 2021, 11:25am UTC](https://discuss.elastic.co/t/question-regarding-tf-idf-implementation/267933/3 "2021-04-19T11:25:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.