Combining match query with "_all" and boosting at query time?

Elastic Stack Elasticsearch
1

Hello there,

is it possible to use a match query which searches in "_all" and combine it
with boosting at query time?

For example: I have a field title and body. Now I'm searching for { "match"
: { "_all" : "foo" } }, but I want to boost hits in title over body at
query time (not at index time).

Greetings from Berlin,
Alex

--

2

Hi Alexander,

Why not use the multi_match query? This allows you to use multiple
fields at same time with different boosts.
Example:
{
"multi_match" : {
"query" : "foo",
"fields" : ["title^1.5", "body^0.5"]
}
}

Since you don't use index time boosts and you want have different
weight per field this seems a better approach to me. You can then also
disable the _all field in your mapping, which will save you disk space
too!

Martijn

On 18 October 2012 14:19, Alexander Pepper alexander.pepper@infopark.de wrote:

Hello there,

is it possible to use a match query which searches in "_all" and combine it
with boosting at query time?

For example: I have a field title and body. Now I'm searching for { "match"
: { "_all" : "foo" } }, but I want to boost hits in title over body at query
time (not at index time).

Greetings from Berlin,
Alex

--

--
Met vriendelijke groet,

Martijn van Groningen

--

3

Hello Martijn,

On Thursday, October 18, 2012 3:27:25 PM UTC+2, Martijn v Groningen wrote:

Why not use the multi_match query? This allows you to use multiple
fields at same time with different boosts.
Example:
{
"multi_match" : {
"query" : "foo",
"fields" : ["title^1.5", "body^0.5"]
}
}

Thank you for your reply. I also thought about using a specific list of
fields. But this was only a simplified example. In reality i have several
fields which are not necessary known to me during query time. Thats why I
want to use the "_all" field, but still boost specific fields. E.g. search
for "foo" in "_all" fields, but boost hits, where it is found in the
attribute "title".

Any ideas?

Greetings from Berlin,
Alex

--

4

So you don't know what fields might by used during query time, but do
you know what fields you possibly want to boost?
If that is the case then I think you still use the match_all query.
Just include the _all field.
{
"multi_match" : {
"query" : "foo",
"fields" : ["_all", "title^2"]
}
}

If a document has a match on either _all field or title field it will
be returned (like a OR). Documents with a hit in title will get an
extra boost of 2.

Martijn

On 18 October 2012 15:34, Alexander Pepper alexander.pepper@infopark.de wrote:

Hello Martijn,

On Thursday, October 18, 2012 3:27:25 PM UTC+2, Martijn v Groningen wrote:

Why not use the multi_match query? This allows you to use multiple
fields at same time with different boosts.
Example:
{
"multi_match" : {
"query" : "foo",
"fields" : ["title^1.5", "body^0.5"]
}
}

Thank you for your reply. I also thought about using a specific list of
fields. But this was only a simplified example. In reality i have several
fields which are not necessary known to me during query time. Thats why I
want to use the "_all" field, but still boost specific fields. E.g. search
for "foo" in "_all" fields, but boost hits, where it is found in the
attribute "title".

Any ideas?

Greetings from Berlin,
Alex

--
Met vriendelijke groet,

Martijn van Groningen

--

5

On Thursday, October 18, 2012 4:47:58 PM UTC+2, Martijn v Groningen wrote:

So you don't know what fields might by used during query time, but do
you know what fields you possibly want to boost?
If that is the case then I think you still use the match_all query.
Just include the _all field.
{
"multi_match" : {
"query" : "foo",
"fields" : ["_all", "title^2"]
}
}

If a document has a match on either _all field or title field it will
be returned (like a OR). Documents with a hit in title will get an
extra boost of 2.

Martijn

Great. This is what I was looking for. Especially the the link to dis_max
in the documentation about multi_match gave me a good clue.

One further question regarding negative boosting: Let's say I want to down
boost hits, where it matches in the title. As far as I can see both with
bool and with dis_max this is not possible, because it either will take the
sum (bool) or the maximum (dis_max).

Bool Query:
{
"bool": {
"should": [
{ "match" : { "_all" : "foo" } },
{ "custom_boost_factor": {
"query": { "match" : { "title" : "foo" } },
"boost_factor": 0.01
}
]
}
}

Dis-Max Query:
{
"dis_max": {
"queries": [
{ "match" : { "_all" : "foo" } },
{ "custom_boost_factor": {
"query": { "match" : { "title" : "foo" } },
"boost_factor": 0.01
}
]
}
}

So my follow up question is: is it possible to lower the score of hits in
the field title while still using the "_all" field?

Greetings from Berlin,
Alex

--

6

Yes, but the query will then be more complex. What might be a good
idea to try is to let the custom_filters_score query wrap your current
query:

Just add a query filter for all fields you want to give a lower boost.

Martijn

On 18 October 2012 17:51, Alexander Pepper alexander.pepper@infopark.de wrote:

On Thursday, October 18, 2012 4:47:58 PM UTC+2, Martijn v Groningen wrote:

So you don't know what fields might by used during query time, but do
you know what fields you possibly want to boost?
If that is the case then I think you still use the match_all query.
Just include the _all field.
{
"multi_match" : {
"query" : "foo",
"fields" : ["_all", "title^2"]
}
}

If a document has a match on either _all field or title field it will
be returned (like a OR). Documents with a hit in title will get an
extra boost of 2.

Martijn

Great. This is what I was looking for. Especially the the link to dis_max in
the documentation about multi_match gave me a good clue.

One further question regarding negative boosting: Let's say I want to down
boost hits, where it matches in the title. As far as I can see both with
bool and with dis_max this is not possible, because it either will take the
sum (bool) or the maximum (dis_max).

Bool Query:
{
"bool": {
"should": [
{ "match" : { "_all" : "foo" } },
{ "custom_boost_factor": {
"query": { "match" : { "title" : "foo" } },
"boost_factor": 0.01
}
]
}
}

Dis-Max Query:
{
"dis_max": {
"queries": [
{ "match" : { "_all" : "foo" } },
{ "custom_boost_factor": {
"query": { "match" : { "title" : "foo" } },
"boost_factor": 0.01
}
]
}
}

So my follow up question is: is it possible to lower the score of hits in
the field title while still using the "_all" field?

Greetings from Berlin,
Alex

--

--
Met vriendelijke groet,

Martijn van Groningen

--

7

On Thu, 2012-10-18 at 20:39 +0200, Martijn v Groningen wrote:

Yes, but the query will then be more complex. What might be a good
idea to try is to let the custom_filters_score query wrap your current
query:
Elasticsearch Platform — Find real-time answers at scale | Elastic

Just add a query filter for all fields you want to give a lower boost.

There's also the "boosting" query that allows you to reduce the score
for docs matching a query

clint

--

8

Hi Clint,

There's also the "boosting" query that allows you to reduce the score
for docs matching a query

Elasticsearch Platform — Find real-time answers at scale | Elastic

clint

As far as I can see, the boosting-query only allows to boost up or down, but not different boosts for different fields. E.G. having field x, y, z, where x should be boosted with 2.0, y with boost 3.0 and z with boost 4.0. Am I right?

Greetings from Berlin,
Alex

--

9

On Fri, 2012-10-19 at 14:07 +0200, Alexander Pepper wrote:

Hi Clint,

There's also the "boosting" query that allows you to reduce the
score
for docs matching a query

Elasticsearch Platform — Find real-time answers at scale | Elastic

clint

As far as I can see, the boosting-query only allows to boost up or
down, but not different boosts for different fields. E.G. having field
x, y, z, where x should be boosted with 2.0, y with boost 3.0 and z
with boost 4.0. Am I right?

Correct. Sorry I must have misread the question

clint

--