Boosting result documents that have a certain key-value combination in a nested strucutre


(Michael Schlenzka) #1

Hi all,

I want to boost documents that have a certain property in a nested
structure (see mapping) with the corresponding value in the nested
structure.

In the example below i got value pairs of a color and a value. Let's say I
want to boost all documents of a query, that have the color blue and I want
to boost it with the corresponding value of blue in each document (so the
boost should be 1 for doc 1 because the value for blue is 1, 5 for doc 3
because the value for blue is 5, ...). What would be a good way to do so?#

I was trying to use function_score or custom_filters_score ... but I am
stuck on accessing the value for a specific color for a specific document
in mvel. Can someone help me on how to access the the value for boosting?
Or has a different solution on how to solve my problem?

Indexmapping:

curl -XPOST "http://localhost:9200/products" -d'
{
"mappings" : {
"product" : {
"properties" : {
"title" : { "type" : "string"},
"uuid" : { "type" : "double"},
"review" : { "type" : "string"},
"colors" : {
"type" : "nested",
"properties": {
"color" : { "type" : "string"},
"value" : {"type" : "integer"}
}
}
}
}
}
}'

Sample Documents:

curl -XPUT "http://localhost:9200/products/product/1" -d'
{
"title": "1",
"uuid": "1",
"review": "test",
"colors": [{
"color": "blue",
"value": 1
}, {
"color": "yellow",
"value": 2
}, {
"color": "red",
"value": 14
}]
}'

curl -XPUT "http://localhost:9200/products/product/2" -d'
{
"title": "2",
"uuid": "2",
"review": "test",
"colors": [{
"colors": "green",
"value": 1
}, {
"color": "white",
"value": 2
}]
}'

curl -XPUT "http://localhost:9200/products/product/3" -d'
{
"title": "3",
"uuid": "3",
"review": "test",
"colors": [{
"color": "blue",
"value": 5
}, {
"color": "green",
"value": 5
}]
}'

curl -XPUT "http://localhost:9200/products/product/4" -d'
{
"title": "4",
"uuid": "4",
"review": "test",
"colors": [{
"color": "green",
"value": 0
}, {
"color": "blue",
"value": 0
}, {
"color": "red",
"value": 0
}, {
"color": "white",
"value": 0
}]
}'

curl -XPUT "http://localhost:9200/products/product/5" -d'
{
"title": "5",
"uuid": "5",
"review": "test"
}'

Query using function_score - problem is that I am unable to access the
value of the corresponding color.

curl -XGET "http://localhost:9200/products/product/_search?pretty=true" -d'
{
"query" : {
"function_score": {
"boost_mode" : "replace",
"query": {
"match" : { "review" : "test"}
},
"boost" : "2",
"functions": [
{
"filter": {
"nested": {
"path": "colors",
"filter": {
"term" :
{"colors.color" : "blue"}
}
}
},
"script_score" : {
"script" : "_score + doc["value"].value"
}
},
{
"script_score" : {
"script" : "_score"
}
}
],
"score_mode" : "first"
}
}
}'

Or I could use custom_filters_score - but then again - how do I get the
value of the corresponding color?

curl -XGET "http://localhost:9200/products/product/_search?pretty=true" -d'
{
"query" : {
"custom_filters_score": {
"query": {
"match" : { "review" : "test"}
},
"filters": [
{
"filter": {
"nested": {
"path": "colors",
"filter": {
"term" :
{"colors.color" : "blue"}
}
}
},
"boost" : 2
}],
"score_mode" : "max"
}
}
}'

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8f24d4b0-29a5-43df-b46a-1103c7d0a6bf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Clinton Gormley) #2

Hiya Michael

The problem with your current attempts is that the nested filter matches on
the color "blue", but then returns the PARENT document, so when you try to
access the colors.value field it is not available.

Instead, you need to run a nested filter, and return the sum of the
colors.value field as the score:

curl -XGET "http://localhost:9200/products/product/_search?pretty=true" -d'
{
"query": {
"bool": {
"disable_coord": true,
"must": {
"match": {
"review": "test"
}
},
"should": {
"nested": {
"path": "colors",
"query": {
"function_score": {
"score_mode": "sum",
"query": {
"term": {
"colors.color": "blue"
}
},
"script_score": {
"script": "doc["value"].value"
}
}
}
}
}
}
}
}'

clint

On 12 March 2014 20:52, Michael Schlenzka michael@schlenzka.com wrote:

Hi all,

I want to boost documents that have a certain property in a nested
structure (see mapping) with the corresponding value in the nested
structure.

In the example below i got value pairs of a color and a value. Let's say I
want to boost all documents of a query, that have the color blue and I want
to boost it with the corresponding value of blue in each document (so the
boost should be 1 for doc 1 because the value for blue is 1, 5 for doc 3
because the value for blue is 5, ...). What would be a good way to do so?#

I was trying to use function_score or custom_filters_score ... but I am
stuck on accessing the value for a specific color for a specific document
in mvel. Can someone help me on how to access the the value for boosting?
Or has a different solution on how to solve my problem?

Indexmapping:

curl -XPOST "http://localhost:9200/products" -d'
{
"mappings" : {
"product" : {
"properties" : {
"title" : { "type" : "string"},
"uuid" : { "type" : "double"},
"review" : { "type" : "string"},
"colors" : {
"type" : "nested",
"properties": {
"color" : { "type" : "string"},
"value" : {"type" : "integer"}
}
}
}
}
}
}'

Sample Documents:

curl -XPUT "http://localhost:9200/products/product/1" -d'
{
"title": "1",
"uuid": "1",
"review": "test",
"colors": [{
"color": "blue",
"value": 1
}, {
"color": "yellow",
"value": 2
}, {
"color": "red",
"value": 14
}]
}'

curl -XPUT "http://localhost:9200/products/product/2" -d'
{
"title": "2",
"uuid": "2",
"review": "test",
"colors": [{
"colors": "green",
"value": 1
}, {
"color": "white",
"value": 2
}]
}'

curl -XPUT "http://localhost:9200/products/product/3" -d'
{
"title": "3",
"uuid": "3",
"review": "test",
"colors": [{
"color": "blue",
"value": 5
}, {
"color": "green",
"value": 5
}]
}'

curl -XPUT "http://localhost:9200/products/product/4" -d'
{
"title": "4",
"uuid": "4",
"review": "test",
"colors": [{
"color": "green",
"value": 0
}, {
"color": "blue",
"value": 0
}, {
"color": "red",
"value": 0
}, {
"color": "white",
"value": 0
}]
}'

curl -XPUT "http://localhost:9200/products/product/5" -d'
{
"title": "5",
"uuid": "5",
"review": "test"
}'

Query using function_score - problem is that I am unable to access the
value of the corresponding color.

curl -XGET "http://localhost:9200/products/product/_search?pretty=true"
-d'
{
"query" : {
"function_score": {
"boost_mode" : "replace",
"query": {
"match" : { "review" : "test"}
},
"boost" : "2",
"functions": [
{
"filter": {
"nested": {
"path": "colors",
"filter": {
"term" :
{"colors.color" : "blue"}
}
}
},
"script_score" : {
"script" : "_score + doc["value"].value"
}
},
{
"script_score" : {
"script" : "_score"
}
}
],
"score_mode" : "first"
}
}
}'

Or I could use custom_filters_score - but then again - how do I get the
value of the corresponding color?

curl -XGET "http://localhost:9200/products/product/_search?pretty=true"
-d'
{
"query" : {
"custom_filters_score": {
"query": {
"match" : { "review" : "test"}
},
"filters": [
{
"filter": {
"nested": {
"path": "colors",
"filter": {
"term" :
{"colors.color" : "blue"}
}
}
},
"boost" : 2
}],
"score_mode" : "max"
}
}
}'

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8f24d4b0-29a5-43df-b46a-1103c7d0a6bf%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/8f24d4b0-29a5-43df-b46a-1103c7d0a6bf%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKTwQRc3msPw9f9QyODLYfOeb%2B-vryyG4iTSmmdmUrxsPg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Michael Schlenzka) #3

Hello Clinton,

Thank you for your answer. Probably didn't describe it that clear, I do not
want the sum of all the values of the key-value-pairs. I want to boost each
document (with a specific key) only with the value for the matching
key/color (e.g. if searching for documents with blue as color each document
should be boosted with the value for blue in this document. Am I correct,
that the sum will give me the sum of all values in the nested value fields
as score/boost? Is the solution to write a script in mvel that iterates
through the nested fields?

Regards

Michael

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8aae16d5-542a-4fa6-9ade-98b3e81ca0d2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Clinton Gormley) #4

On 12 March 2014 23:32, Michael Schlenzka michael@schlenzka.com wrote:

I do not want the sum of all the values of the key-value-pairs. I want to
boost each document (with a specific key) only with the value for the
matching key/color (e.g. if searching for documents with blue as color each
document should be boosted with the value for blue in this document. Am I
correct, that the sum will give me the sum of all values in the nested
value fields as score/boost?

This is exactly what my query achieves. The nested query matches nested
docs which have color=blue, and adds up the value for just those docs. It
returns this sum as the _score from that query.

The bool query then adds the _score from the nested query to the _score
from the review=test query and returns that as the overall _score per doc.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKRJx9L2izFf1csZmdzce_oSSzcRDW1kttkhkARSEW%3D2YQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #5

Have you tried term boosting?

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#_boosting

Jörg

On Wed, Mar 12, 2014 at 8:52 PM, Michael Schlenzka michael@schlenzka.comwrote:

Hi all,

I want to boost documents that have a certain property in a nested
structure (see mapping) with the corresponding value in the nested
structure.

In the example below i got value pairs of a color and a value. Let's say I
want to boost all documents of a query, that have the color blue and I want
to boost it with the corresponding value of blue in each document (so the
boost should be 1 for doc 1 because the value for blue is 1, 5 for doc 3
because the value for blue is 5, ...). What would be a good way to do so?#

I was trying to use function_score or custom_filters_score ... but I am
stuck on accessing the value for a specific color for a specific document
in mvel. Can someone help me on how to access the the value for boosting?
Or has a different solution on how to solve my problem?

Indexmapping:

curl -XPOST "http://localhost:9200/products" -d'
{
"mappings" : {
"product" : {
"properties" : {
"title" : { "type" : "string"},
"uuid" : { "type" : "double"},
"review" : { "type" : "string"},
"colors" : {
"type" : "nested",
"properties": {
"color" : { "type" : "string"},
"value" : {"type" : "integer"}
}
}
}
}
}
}'

Sample Documents:

curl -XPUT "http://localhost:9200/products/product/1" -d'
{
"title": "1",
"uuid": "1",
"review": "test",
"colors": [{
"color": "blue",
"value": 1
}, {
"color": "yellow",
"value": 2
}, {
"color": "red",
"value": 14
}]
}'

curl -XPUT "http://localhost:9200/products/product/2" -d'
{
"title": "2",
"uuid": "2",
"review": "test",
"colors": [{
"colors": "green",
"value": 1
}, {
"color": "white",
"value": 2
}]
}'

curl -XPUT "http://localhost:9200/products/product/3" -d'
{
"title": "3",
"uuid": "3",
"review": "test",
"colors": [{
"color": "blue",
"value": 5
}, {
"color": "green",
"value": 5
}]
}'

curl -XPUT "http://localhost:9200/products/product/4" -d'
{
"title": "4",
"uuid": "4",
"review": "test",
"colors": [{
"color": "green",
"value": 0
}, {
"color": "blue",
"value": 0
}, {
"color": "red",
"value": 0
}, {
"color": "white",
"value": 0
}]
}'

curl -XPUT "http://localhost:9200/products/product/5" -d'
{
"title": "5",
"uuid": "5",
"review": "test"
}'

Query using function_score - problem is that I am unable to access the
value of the corresponding color.

curl -XGET "http://localhost:9200/products/product/_search?pretty=true"
-d'
{
"query" : {
"function_score": {
"boost_mode" : "replace",
"query": {
"match" : { "review" : "test"}
},
"boost" : "2",
"functions": [
{
"filter": {
"nested": {
"path": "colors",
"filter": {
"term" :
{"colors.color" : "blue"}
}
}
},
"script_score" : {
"script" : "_score + doc["value"].value"
}
},
{
"script_score" : {
"script" : "_score"
}
}
],
"score_mode" : "first"
}
}
}'

Or I could use custom_filters_score - but then again - how do I get the
value of the corresponding color?

curl -XGET "http://localhost:9200/products/product/_search?pretty=true"
-d'
{
"query" : {
"custom_filters_score": {
"query": {
"match" : { "review" : "test"}
},
"filters": [
{
"filter": {
"nested": {
"path": "colors",
"filter": {
"term" :
{"colors.color" : "blue"}
}
}
},
"boost" : 2
}],
"score_mode" : "max"
}
}
}'

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8f24d4b0-29a5-43df-b46a-1103c7d0a6bf%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/8f24d4b0-29a5-43df-b46a-1103c7d0a6bf%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHrqPvHfCrjZ0tyTRwXeF7eEN%3DhrG57b8nd36%2Be4jMHVg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Clinton Gormley) #6

On 13 March 2014 11:35, joergprante@gmail.com joergprante@gmail.com wrote:

Have you tried term boosting?

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#_boosting

This won't help him boost by the value in the value field associated with
each color=blue nested document.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKRXcgtaAwvYjscO4ByHpJUXkjzMVPCZua6Y2j_m-MUovQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #7