Question on breaking change (boost) in 1.0.0.RC1 release

Amit_Soni · January 16, 2014, 7:47pm

Hi all - I have been going through the list of breaking changes in
1.0.0.RC1 and have a question regarding boosting of documents. I see
that "*Support
for document boosting via the _boost field has been removed from Lucene and
is deprecated in Elasticsearch as of v1.0.0.RC1. *"

http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/mapping-boost-field.html#function-score-instead-of-boost

Since I didnt really understand if there is a way one can still boost
documents during indexing, I wanted to check with this group. So if I
correctly understand there would be no mechanism to specify boost value of
the documents during indexing time. Is that right?

-Amit.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQJbeK0y%3DwzpKBQ7CMP2t_Et8j%3D4QXY6ggLisYLcKADZbQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Ivan · January 16, 2014, 7:54pm

Judging by the commits, the functionality was only deprecated, not removed.

github.com/elastic/elasticsearch

Deprecate document boost

opened 10:09AM - 09 Jan 14 UTC

closed 03:04PM - 09 Jan 14 UTC

javanna

>breaking v1.0.0.RC1

The document boost is a nice feature but since it was removed from lucene 4.0, t…he way it works in elasticsearch is by adding fields boosts to each field, multiplying it with the original field boost. Here is the interesting commit: https://github.com/elasticsearch/elasticsearch/commit/c60f20413b299e4d9ea0a5fa3e24381e90d914b8#diff-7117c679a1ca0d5002c0c9b9ef8bad16 . That is not exactly how the document boost should work, it has downsides and the same result can be obtained using function_score. For the above reasons we are going to deprecate the document boost.

I believe there are many use cases where it makes sense to boost a document
at index time. The process only occurs once instead of every time during
queries.

That said, as the github issue says, document boosts are very confusing.
What it basically does behind the scenes is changing the field norm for
every field in the document. If you disable norms on a field, then that
document cannot boost on that field. The scoring becomes erratic and hard
to explain. You cannot disable length normalization on fields since that
would require the field norms, which are needed by the document boosts.
Overall it believe it is better to switch to custom scoring at query time.

Cheers,

Ivan

On Thu, Jan 16, 2014 at 11:47 AM, Amit Soni amitsoni29@gmail.com wrote:

Hi all - I have been going through the list of breaking changes in
1.0.0.RC1 and have a question regarding boosting of documents. I see that "*Support
for document boosting via the _boost field has been removed from Lucene and
is deprecated in Elasticsearch as of v1.0.0.RC1. *"

Elasticsearch Platform — Find real-time answers at scale | Elastic

Since I didnt really understand if there is a way one can still boost
documents during indexing, I wanted to check with this group. So if I
correctly understand there would be no mechanism to specify boost value of
the documents during indexing time. Is that right?

-Amit.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAAOGaQJbeK0y%3DwzpKBQ7CMP2t_Et8j%3D4QXY6ggLisYLcKADZbQ%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBHBCM7f%2BpZL7ZZ7d7HFzxqTr%3DO5begP%2BcK1JD7Rp3u2w%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

jprante · January 16, 2014, 10:57pm

You can continue to set up your document boost field (e.g. a numeric field
named "boost") with a document boost value, and use function score to use
it as a boost factor. That is, a function score script knows how to boost
correctly:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_boost_factor

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGfj-C1w2B7mhFFRNJSG1SfAVPwZDnqUR08nh1XnzjytA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Amit_Soni · January 31, 2014, 8:27am

Thanks so much Jörg and Ivan.
Do you think I can use function score with any query? I use simple query
string query and wondering how this can be used along with that?

-Amit.

On Thu, Jan 16, 2014 at 2:57 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

You can continue to set up your document boost field (e.g. a numeric field
named "boost") with a document boost value, and use function score to use
it as a boost factor. That is, a function score script knows how to boost
correctly:

Elasticsearch Platform — Find real-time answers at scale | Elastic

Jörg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGfj-C1w2B7mhFFRNJSG1SfAVPwZDnqUR08nh1XnzjytA%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQK%2BB9L5HQ6TnWY-Z09pR5p-k5L0apm6yaGmW1poGNFn%3Dg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

jprante · January 31, 2014, 9:31am

See this full example of document boosting with function score query, you
can use any query you like.

gist.github.com

https://gist.github.com/jprante/8728976

doc-boost-function-score.sh


curl -XDELETE 'localhost:9200/test'

curl -XPUT 'localhost:9200/test/doc/1' -d '
{   
    "sentence" : "less important",
    "boost" : 0.5
}
'

This file has been truncated. show original

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH9FY2xbNGK70Tg9%2B0k75TJmvwkhrcAUuWNDDNkOr%2BGqw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Amit_Soni · February 3, 2014, 2:20am

Thanks much Jörg, this is super helpful. I didn't realize we could wrap a
query inside function score query.

-Amit.

On Fri, Jan 31, 2014 at 1:31 AM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

See this full example of document boosting with function score query, you
can use any query you like.

Document boosting with function score query · GitHub

Jörg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH9FY2xbNGK70Tg9%2B0k75TJmvwkhrcAUuWNDDNkOr%2BGqw%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAOGaQLeSNVQ_1xb1zUJHEYs2koifrOxsRGZXaeO%3DifUZGF8zQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.