Custom Score Query not working in elasticsearch-0.90.3


(dark_shadow) #1

Hi,

I'm facing an issue in Elasticsearch-0.90.3 in my java code. I'm writing
following query:

String script = "_score * (doc['po'].empty ? 1 : doc['po'].value == 0.0 ? 1
: doc['po'].value)";
QueryBuilder queryBuilder = QueryBuilders.
customScoreQuery(QueryBuilders.queryString(query)
.field("text",30)
.field("ad")
.field("st")
.field("cn")
.field("co")

.defaultOperator(Operator.AND)).script(script);

But the above script calculates same score for similar documents even if
they have a different po value. The issue is it's not taking doc['po'] into
consideration.

Following is the output of my explain parameter:

{category=Hotel, text=hotels in ranchi, count=45.0,
_id=525472d7d4a769f431649936, location={lon=85.33333, lat=23.35}, po=8.8}
1195.7068 = custom score, product of:
1195.7068 = script score function: composed of:
239.14136 = sum of:
215.63971 = max of:
215.63971 = sum of:
18.701233 = weight(text:ho in 170585) [PerFieldSimilarity],
result of:
18.701233 = score(doc=170585,freq=1.0 = termFreq=1.0
), product of:
0.27964544 = queryWeight, product of:
2.7864501 = idf(docFreq=960416, maxDocs=5731988)
0.10035904 = queryNorm
66.8748 = fieldWeight in 170585, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
2.7864501 = idf(docFreq=960416, maxDocs=5731988)
24.0 = fieldNorm(doc=170585)
22.496681 = weight(text:hot in 170585) [PerFieldSimilarity],
result of:
22.496681 = score(doc=170585,freq=1.0 = termFreq=1.0
), product of:
0.30671278 = queryWeight, product of:
3.056155 = idf(docFreq=733378, maxDocs=5731988)
0.10035904 = queryNorm
73.34772 = fieldWeight in 170585, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
3.056155 = idf(docFreq=733378, maxDocs=5731988)
24.0 = fieldNorm(doc=170585)
22.589373 = weight(text:hote in 170585) [PerFieldSimilarity],
result of:
22.589373 = score(doc=170585,freq=1.0 = termFreq=1.0
), product of:
0.307344 = queryWeight, product of:
3.0624444 = idf(docFreq=728780, maxDocs=5731988)
0.10035904 = queryNorm
73.498665 = fieldWeight in 170585, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
3.0624444 = idf(docFreq=728780, maxDocs=5731988)
24.0 = fieldNorm(doc=170585)
22.600088 = weight(text:hotel in 170585) [PerFieldSimilarity],
result of:
22.600088 = score(doc=170585,freq=1.0 = termFreq=1.0
), product of:
0.30741686 = queryWeight, product of:
3.0631707 = idf(docFreq=728251, maxDocs=5731988)
0.10035904 = queryNorm
73.5161 = fieldWeight in 170585, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
3.0631707 = idf(docFreq=728251, maxDocs=5731988)
24.0 = fieldNorm(doc=170585)
129.25233 = weight(text:hotels in 170585) [PerFieldSimilarity],
result of:
129.25233 = score(doc=170585,freq=1.0 = termFreq=1.0
), product of:
0.73517686 = queryWeight, product of:
7.3254676 = idf(docFreq=10260, maxDocs=5731988)
0.10035904 = queryNorm
175.81122 = fieldWeight in 170585, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
7.3254676 = idf(docFreq=10260, maxDocs=5731988)
24.0 = fieldNorm(doc=170585)
23.50164 = max of:
23.50164 = weight(text:in^30.0 in 170585) [PerFieldSimilarity],
result of:
23.50164 = score(doc=170585,freq=1.0 = termFreq=1.0
), product of:
0.31348857 = queryWeight, product of:
30.0 = boost
3.1236706 = idf(docFreq=685498, maxDocs=5731988)
0.0033453014 = queryNorm
74.968094 = fieldWeight in 170585, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
3.1236706 = idf(docFreq=685498, maxDocs=5731988)
24.0 = fieldNorm(doc=170585)
1.0 = queryBoost

{category=Hotel, text=hotels in kerala, count=5.0,
_id=525472d7d4a769f4316499ca, location={lon=93.95, lat=24.81667},
po=9.228571428571428}
1195.7068 = custom score, product of:
1195.7068 = script score function: composed of:
239.14136 = sum of:
215.63971 = max of:
215.63971 = sum of:
18.701233 = weight(text:ho in 170733) [PerFieldSimilarity],
result of:
18.701233 = score(doc=170733,freq=1.0 = termFreq=1.0
), product of:
0.27964544 = queryWeight, product of:
2.7864501 = idf(docFreq=960416, maxDocs=5731988)
0.10035904 = queryNorm
66.8748 = fieldWeight in 170733, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
2.7864501 = idf(docFreq=960416, maxDocs=5731988)
24.0 = fieldNorm(doc=170733)
22.496681 = weight(text:hot in 170733) [PerFieldSimilarity],
result of:
22.496681 = score(doc=170733,freq=1.0 = termFreq=1.0
), product of:
0.30671278 = queryWeight, product of:
3.056155 = idf(docFreq=733378, maxDocs=5731988)
0.10035904 = queryNorm
73.34772 = fieldWeight in 170733, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
3.056155 = idf(docFreq=733378, maxDocs=5731988)
24.0 = fieldNorm(doc=170733)
22.589373 = weight(text:hote in 170733) [PerFieldSimilarity],
result of:
22.589373 = score(doc=170733,freq=1.0 = termFreq=1.0
), product of:
0.307344 = queryWeight, product of:
3.0624444 = idf(docFreq=728780, maxDocs=5731988)
0.10035904 = queryNorm
73.498665 = fieldWeight in 170733, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
3.0624444 = idf(docFreq=728780, maxDocs=5731988)
24.0 = fieldNorm(doc=170733)
22.600088 = weight(text:hotel in 170733) [PerFieldSimilarity],
result of:
22.600088 = score(doc=170733,freq=1.0 = termFreq=1.0
), product of:
0.30741686 = queryWeight, product of:
3.0631707 = idf(docFreq=728251, maxDocs=5731988)
0.10035904 = queryNorm
73.5161 = fieldWeight in 170733, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
3.0631707 = idf(docFreq=728251, maxDocs=5731988)
24.0 = fieldNorm(doc=170733)
129.25233 = weight(text:hotels in 170733) [PerFieldSimilarity],
result of:
129.25233 = score(doc=170733,freq=1.0 = termFreq=1.0
), product of:
0.73517686 = queryWeight, product of:
7.3254676 = idf(docFreq=10260, maxDocs=5731988)
0.10035904 = queryNorm
175.81122 = fieldWeight in 170733, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
7.3254676 = idf(docFreq=10260, maxDocs=5731988)
24.0 = fieldNorm(doc=170733)
23.50164 = max of:
23.50164 = weight(text:in^30.0 in 170733) [PerFieldSimilarity],
result of:
23.50164 = score(doc=170733,freq=1.0 = termFreq=1.0
), product of:
0.31348857 = queryWeight, product of:
30.0 = boost
3.1236706 = idf(docFreq=685498, maxDocs=5731988)
0.0033453014 = queryNorm
74.968094 = fieldWeight in 170733, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
3.1236706 = idf(docFreq=685498, maxDocs=5731988)
24.0 = fieldNorm(doc=170733)
1.0 = queryBoost

After going deep into the matter, I found that for every document doc['po']
is empty, that's it is setting script = _score * 1 (always).

But all my docs contain this field with a double value. Here is how my
mapping looks like:

curl -XPUT 'localhost:9200/auto_index/
autocomplete/_mapping' -d '{
"autocomplete":{
"_boost" : {
"name" : "po",
"null_value" : 4.0
},
"properties": {
"ad": {
"type": "string",
"search_analyzer" : "str_search_analyzer",
"index_analyzer" : "str_index_analyzer"
},
"category": {
"type": "string",
"include_in_all" : false
},
"cn": {
"type": "string",
"search_analyzer" : "str_search_analyzer",
"index_analyzer" : "str_index_analyzer"
},
"ctype": {
"type": "string",
"search_analyzer" : "keyword",
"index_analyzer" : "keyword"
},
"eid": {
"type": "string",
"include_in_all" : false
},
"st": {
"type": "string",
"search_analyzer" : "str_search_analyzer",
"index_analyzer" : "str_index_analyzer"
},
"co": {
"type": "string",
"include_in_all" : false
},
"st": {
"type": "string",
"search_analyzer" : "str_search_analyzer",
"index_analyzer" : "str_index_analyzer"
},
"co": {
"type": "string",
"search_analyzer" : "str_search_analyzer",
"index_analyzer" : "str_index_analyzer"
},
"po": {
"type": "double",
"boost": 4.0
},
"en":{
"type": "boolean"
},
"_oid":{
"type": "long"
},
"text": {
"type": "string",
"search_analyzer" : "str_search_analyzer",
"index_analyzer" : "str_index_analyzer"
},
"url": {
"type": "string"
}
}
}
}'

Why Elasticsearch is not able to access/find the value of "po" field ? Is
it not supported in Elasticsearch-0.90.3 ?

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b3b0224e-0e9e-4671-b7bf-147a74656d2a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Binh Ly) #2

Coder, I can duplicate your problem. I am trying to find out if this is a
bug, or it is by design and will get back to you shortly. Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4c6625f0-299c-441a-8c44-826e543054e7%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Binh Ly) #3

Coder, I've verified that this is a bug. However, the "_boost" field will
be deprecated in 1.0 in favor of the function_score query
(http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_using_function_score).

For now, what you can do to fix this is to remove the "_boost" part from
your mapping and reindex your data and you should be able to access the
"po" field after that.

Remove this part from your mapping:

   "_boost" : {
        "name" : "po", 
        "null_value" : 4.0
   },

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a1894a05-4195-44eb-aa62-80f173a58a99%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ivan Brusic) #4

I fixed this bug a while ago and it should be fixed in versions 0.90.6 and
higher.


Ivan

On Thu, Jan 30, 2014 at 11:51 AM, Binh Ly binh@hibalo.com wrote:

Coder, I've verified that this is a bug. However, the "_boost" field will
be deprecated in 1.0 in favor of the function_score query (
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_using_function_score).

For now, what you can do to fix this is to remove the "_boost" part from
your mapping and reindex your data and you should be able to access the
"po" field after that.

Remove this part from your mapping:

   "_boost" : {
        "name" : "po",
        "null_value" : 4.0
   },

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a1894a05-4195-44eb-aa62-80f173a58a99%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAeYwN-%3DLNyaGRpMfuQuW8ytk0nJDtdv2gpKgGThO1YWg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ivan Brusic) #5

BTW, if you do not want to upgrade, you can also simply maintain two
fields. One to use as a document boost (the _boost) and another to use for
scripting/sorting/faceting.

--
Ivan

On Thu, Jan 30, 2014 at 4:00 PM, Ivan Brusic ivan@brusic.com wrote:

I fixed this bug a while ago and it should be fixed in versions 0.90.6 and
higher.

https://github.com/elasticsearch/elasticsearch/issues/3752
https://github.com/elasticsearch/elasticsearch/pull/2913

Ivan

On Thu, Jan 30, 2014 at 11:51 AM, Binh Ly binh@hibalo.com wrote:

Coder, I've verified that this is a bug. However, the "_boost" field will
be deprecated in 1.0 in favor of the function_score query (
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_using_function_score).

For now, what you can do to fix this is to remove the "_boost" part from
your mapping and reindex your data and you should be able to access the
"po" field after that.

Remove this part from your mapping:

   "_boost" : {
        "name" : "po",
        "null_value" : 4.0
   },

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a1894a05-4195-44eb-aa62-80f173a58a99%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDY1nxQB_cYu%2B7Ph8Osai3PMNTUyKQuW8zL2z9OYxYAOQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(dark_shadow) #6

Binh,

Thanks for the help. I fixed my scoring issue but loat my indexed time
boosting. I tried using boosting at indexing using some other fields but
results are not very satisfactory. I was using custom score so that I can
get documents matching a search phrase with higher value of "po" field on
the top but things are not going that way. Is there any way of modifying my
current query mechanism so that I can get what I expect.

I want to search in a way which gives me docs which contains some part or
the whole text which is searched by user instead right now I'm getting docs
which contains words of the text only.

On Fri, Jan 31, 2014 at 5:30 AM, Ivan Brusic ivan@brusic.com wrote:

I fixed this bug a while ago and it should be fixed in versions 0.90.6 and
higher.

https://github.com/elasticsearch/elasticsearch/issues/3752
https://github.com/elasticsearch/elasticsearch/pull/2913

Ivan

On Thu, Jan 30, 2014 at 11:51 AM, Binh Ly binh@hibalo.com wrote:

Coder, I've verified that this is a bug. However, the "_boost" field will
be deprecated in 1.0 in favor of the function_score query (
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_using_function_score).

For now, what you can do to fix this is to remove the "_boost" part from
your mapping and reindex your data and you should be able to access the
"po" field after that.

Remove this part from your mapping:

   "_boost" : {
        "name" : "po",
        "null_value" : 4.0
   },

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a1894a05-4195-44eb-aa62-80f173a58a99%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAeYwN-%3DLNyaGRpMfuQuW8ytk0nJDtdv2gpKgGThO1YWg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAVTvp6-YkRL7qA6WjvTnOQEu-SDV07MCXWn0WntAH8q%2Bd63iA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(dark_shadow) #7

Ivan, I tried using different field for script and boost. script seems to
be working fine but boost is not working since I'm not able to see any
factor from _boost in explain api. Can you please tell me how can I make
use of boost ?

On Fri, Jan 31, 2014 at 5:53 AM, Ivan Brusic ivan@brusic.com wrote:

BTW, if you do not want to upgrade, you can also simply maintain two
fields. One to use as a document boost (the _boost) and another to use for
scripting/sorting/faceting.

--
Ivan

On Thu, Jan 30, 2014 at 4:00 PM, Ivan Brusic ivan@brusic.com wrote:

I fixed this bug a while ago and it should be fixed in versions 0.90.6
and higher.

https://github.com/elasticsearch/elasticsearch/issues/3752
https://github.com/elasticsearch/elasticsearch/pull/2913

Ivan

On Thu, Jan 30, 2014 at 11:51 AM, Binh Ly binh@hibalo.com wrote:

Coder, I've verified that this is a bug. However, the "_boost" field
will be deprecated in 1.0 in favor of the function_score query (
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_using_function_score).

For now, what you can do to fix this is to remove the "_boost" part from
your mapping and reindex your data and you should be able to access the
"po" field after that.

Remove this part from your mapping:

   "_boost" : {
        "name" : "po",
        "null_value" : 4.0
   },

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a1894a05-4195-44eb-aa62-80f173a58a99%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDY1nxQB_cYu%2B7Ph8Osai3PMNTUyKQuW8zL2z9OYxYAOQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAVTvp6bcp5Zy9a5z8GFz1r_-9raub5NDsHAE%2BacEegaOrW%2BoQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ivan Brusic) #8

Here is part of something I wrote on another thread just 20 minutes ago:

Index-time boosts are encoded inside the field norms. You should see a
difference inside each field norm. If you have omitted norms, then you will
not have any boosts on that field. The field norm is also lossly since it
uses only 1-byte. Some of the many reasons to switch from document time
boosts.

There is no reason to use both document boosts and custom scores. If you
are already using a score function, just apply the boost at that level.
Either way, you will not see the boost value directly.

Ivan

On Fri, Jan 31, 2014 at 8:13 AM, Mukul Gupta mukulnitkkr@gmail.com wrote:

Ivan, I tried using different field for script and boost. script seems to
be working fine but boost is not working since I'm not able to see any
factor from _boost in explain api. Can you please tell me how can I make
use of boost ?

On Fri, Jan 31, 2014 at 5:53 AM, Ivan Brusic ivan@brusic.com wrote:

BTW, if you do not want to upgrade, you can also simply maintain two
fields. One to use as a document boost (the _boost) and another to use for
scripting/sorting/faceting.

--
Ivan

On Thu, Jan 30, 2014 at 4:00 PM, Ivan Brusic ivan@brusic.com wrote:

I fixed this bug a while ago and it should be fixed in versions 0.90.6
and higher.

https://github.com/elasticsearch/elasticsearch/issues/3752
https://github.com/elasticsearch/elasticsearch/pull/2913

Ivan

On Thu, Jan 30, 2014 at 11:51 AM, Binh Ly binh@hibalo.com wrote:

Coder, I've verified that this is a bug. However, the "_boost" field
will be deprecated in 1.0 in favor of the function_score query (
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_using_function_score).

For now, what you can do to fix this is to remove the "_boost" part
from your mapping and reindex your data and you should be able to access
the "po" field after that.

Remove this part from your mapping:

   "_boost" : {
        "name" : "po",
        "null_value" : 4.0
   },

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a1894a05-4195-44eb-aa62-80f173a58a99%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDY1nxQB_cYu%2B7Ph8Osai3PMNTUyKQuW8zL2z9OYxYAOQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAAVTvp6bcp5Zy9a5z8GFz1r_-9raub5NDsHAE%2BacEegaOrW%2BoQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQD3vOZ-skoK6k02mrCE%3Df-T8yyz3d1pRQ%2Bzwi2OV3SSrg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(dark_shadow) #9

I agree that modifying the score of a doc gives me same effect as the
document boost but somehow I'm not finding the results very satisfactory.
I'm thinking may be using _boost for the entire doc can give me some
advantage. Also, I'm talking about _boost not the boost per field,
something like this:

   "_boost" : {
        "name" : "po",
        "null_value" : 4.0
   }

Please correct me if _boost is same as boost which I guess is not as far as
I have understood.

Thanks

On Fri, Jan 31, 2014 at 9:52 PM, Ivan Brusic ivan@brusic.com wrote:

Here is part of something I wrote on another thread just 20 minutes ago:

Index-time boosts are encoded inside the field norms. You should see a
difference inside each field norm. If you have omitted norms, then you will
not have any boosts on that field. The field norm is also lossly since it
uses only 1-byte. Some of the many reasons to switch from document time
boosts.

There is no reason to use both document boosts and custom scores. If you
are already using a score function, just apply the boost at that level.
Either way, you will not see the boost value directly.

Ivan

On Fri, Jan 31, 2014 at 8:13 AM, Mukul Gupta mukulnitkkr@gmail.comwrote:

Ivan, I tried using different field for script and boost. script seems to
be working fine but boost is not working since I'm not able to see any
factor from _boost in explain api. Can you please tell me how can I make
use of boost ?

On Fri, Jan 31, 2014 at 5:53 AM, Ivan Brusic ivan@brusic.com wrote:

BTW, if you do not want to upgrade, you can also simply maintain two
fields. One to use as a document boost (the _boost) and another to use for
scripting/sorting/faceting.

--
Ivan

On Thu, Jan 30, 2014 at 4:00 PM, Ivan Brusic ivan@brusic.com wrote:

I fixed this bug a while ago and it should be fixed in versions 0.90.6
and higher.

https://github.com/elasticsearch/elasticsearch/issues/3752
https://github.com/elasticsearch/elasticsearch/pull/2913

Ivan

On Thu, Jan 30, 2014 at 11:51 AM, Binh Ly binh@hibalo.com wrote:

Coder, I've verified that this is a bug. However, the "_boost" field
will be deprecated in 1.0 in favor of the function_score query (
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_using_function_score).

For now, what you can do to fix this is to remove the "_boost" part
from your mapping and reindex your data and you should be able to access
the "po" field after that.

Remove this part from your mapping:

   "_boost" : {
        "name" : "po",
        "null_value" : 4.0
   },

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a1894a05-4195-44eb-aa62-80f173a58a99%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDY1nxQB_cYu%2B7Ph8Osai3PMNTUyKQuW8zL2z9OYxYAOQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAAVTvp6bcp5Zy9a5z8GFz1r_-9raub5NDsHAE%2BacEegaOrW%2BoQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQD3vOZ-skoK6k02mrCE%3Df-T8yyz3d1pRQ%2Bzwi2OV3SSrg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAVTvp65GDe79Gf%2B4sz23T_Yr4s4b%2BZ5eJZUsy%3DZ409mfG0jXQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ivan Brusic) #10

With Lucene 4 and higher, there is no true document boost. What is
happening behind the scenes is that the document boost is being encoded
inside the field norm of every field in the document. If you want to see
the boost, you need to examine each fieldnorm. It would look something like
this

{
"value": 10,
"description": "fieldNorm(doc=1115396)"
}

In this example case, the boost was set to 99. 10==99. Confusing isn't it?
It does translate to a boost of 99 (for that one field), but it is
confusing to read. One of the reasons not to use document boosts.

Cheers,

Ivan

On Fri, Jan 31, 2014 at 8:28 AM, Mukul Gupta mukulnitkkr@gmail.com wrote:

I agree that modifying the score of a doc gives me same effect as the
document boost but somehow I'm not finding the results very satisfactory.
I'm thinking may be using _boost for the entire doc can give me some
advantage. Also, I'm talking about _boost not the boost per field,
something like this:

   "_boost" : {
        "name" : "po",
        "null_value" : 4.0
   }

Please correct me if _boost is same as boost which I guess is not as far
as I have understood.

Thanks

On Fri, Jan 31, 2014 at 9:52 PM, Ivan Brusic ivan@brusic.com wrote:

Here is part of something I wrote on another thread just 20 minutes ago:

Index-time boosts are encoded inside the field norms. You should see a
difference inside each field norm. If you have omitted norms, then you will
not have any boosts on that field. The field norm is also lossly since it
uses only 1-byte. Some of the many reasons to switch from document time
boosts.

There is no reason to use both document boosts and custom scores. If you
are already using a score function, just apply the boost at that level.
Either way, you will not see the boost value directly.

Ivan

On Fri, Jan 31, 2014 at 8:13 AM, Mukul Gupta mukulnitkkr@gmail.comwrote:

Ivan, I tried using different field for script and boost. script seems
to be working fine but boost is not working since I'm not able to see any
factor from _boost in explain api. Can you please tell me how can I make
use of boost ?

On Fri, Jan 31, 2014 at 5:53 AM, Ivan Brusic ivan@brusic.com wrote:

BTW, if you do not want to upgrade, you can also simply maintain two
fields. One to use as a document boost (the _boost) and another to use for
scripting/sorting/faceting.

--
Ivan

On Thu, Jan 30, 2014 at 4:00 PM, Ivan Brusic ivan@brusic.com wrote:

I fixed this bug a while ago and it should be fixed in versions 0.90.6
and higher.

https://github.com/elasticsearch/elasticsearch/issues/3752
https://github.com/elasticsearch/elasticsearch/pull/2913

Ivan

On Thu, Jan 30, 2014 at 11:51 AM, Binh Ly binh@hibalo.com wrote:

Coder, I've verified that this is a bug. However, the "_boost" field
will be deprecated in 1.0 in favor of the function_score query (
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_using_function_score).

For now, what you can do to fix this is to remove the "_boost" part
from your mapping and reindex your data and you should be able to access
the "po" field after that.

Remove this part from your mapping:

   "_boost" : {
        "name" : "po",
        "null_value" : 4.0
   },

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a1894a05-4195-44eb-aa62-80f173a58a99%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDY1nxQB_cYu%2B7Ph8Osai3PMNTUyKQuW8zL2z9OYxYAOQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAAVTvp6bcp5Zy9a5z8GFz1r_-9raub5NDsHAE%2BacEegaOrW%2BoQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQD3vOZ-skoK6k02mrCE%3Df-T8yyz3d1pRQ%2Bzwi2OV3SSrg%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAAVTvp65GDe79Gf%2B4sz23T_Yr4s4b%2BZ5eJZUsy%3DZ409mfG0jXQ%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDXkwvM2QLcAOkGG4oOVkHupjRCyxcgeZQN00SJ%3DAPxmg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #11