TF/IDF wihout TF

Elastic Stack Elasticsearch
1

How to turn TF off (that is a single term in a field is the same as
multiple ones) in scoring (keeping IDF and field length)?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d527960c-1407-4f61-b058-2baab38b4957%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

You can turn off tf, but you also have to turn off positions at the same
time. (There is no positions without term frequency). This implies you
won't be able to do a phrase query. This leaves IDF and norms as the main
scoring component

See the "index_options" here for disabling term freq

-Doug

On Sat, Dec 27, 2014 at 10:10 AM, Andrew Gaydenko <andrew.gaydenko@gmail.com

wrote:

How to turn TF off (that is a single term in a field is the same as
multiple ones) in scoring (keeping IDF and field length)?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d527960c-1407-4f61-b058-2baab38b4957%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d527960c-1407-4f61-b058-2baab38b4957%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Doug Turnbull
Search & Big Data Architect
OpenSource Connections http://o19s.com

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALG6HL9ei6T1sAE30YXs%3D8QtBd%3DwuymfyOP0XUnaqW446HoQSQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Thanks! Having norms on and docs for index_options will we keep scoring on
the field length? - that is "abc" in "abc" has more score than in "abd
def". In other words, do norms include field-length-based scoring factor?

On Saturday, December 27, 2014 6:35:39 PM UTC+3, Doug Turnbull wrote:

You can turn off tf, but you also have to turn off positions at the same
time. (There is no positions without term frequency). This implies you
won't be able to do a phrase query. This leaves IDF and norms as the main
scoring component

See the "index_options" here for disabling term freq

Elasticsearch Platform — Find real-time answers at scale | Elastic

-Doug

On Sat, Dec 27, 2014 at 10:10 AM, Andrew Gaydenko <andrew....@gmail.com
<javascript:>> wrote:

How to turn TF off (that is a single term in a field is the same as
multiple ones) in scoring (keeping IDF and field length)?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d527960c-1407-4f61-b058-2baab38b4957%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d527960c-1407-4f61-b058-2baab38b4957%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Doug Turnbull
Search & Big Data Architect
OpenSource Connections http://o19s.com

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a6792674-1fcd-4c96-b409-f1f8c6d9b860%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

You can "disable" TF by multiplying it with itself. This requires adding a configuration in the base similarity module I think. I have not tried this yet, so please let us know if this will work!

/JZ

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a15814ff-f02e-472b-9d70-dcd9a5d860b5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.