Bool Queries and MUST/SHOULD combinations

Hi,

Below is my data and the two queries that I tested, first one failing and
the latter working. I start to believe that if one wants to combine several
SHOULD and MUST filters, the outer one must always be SHOULD. Is this a
correct assumption? In our application, we have much more complex situation
with several filters within each MUST and SHOULD. And lastly, where should
place a MUST_NOT in this case?

Many thanks.

Here is my data:

_index,_type,_id,_score,_source.id,_source.type,_source.valueType,_source.sentence,_source.location
"test","var","0","1","0","study","text","Lorem text is jumbled","spain"
"test","var","1","1","1","study","text","bla bla bla","spain"
"test","var","2","1","2","schema","decimal","ipsum","germany"
"test","var","3","1","3","study","integer","lorem","france"

Here is my FAILING query:

{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": {
"terms": {
"location": [
"germany"
]
}
},
"should": {
"terms": {
"valueType": [
"integer"
]
}
}
}
}
}
}
}

Here is my WORKING query returning IDs 2 and 3:

{
"query": {
"bool": {
"should": [
{
"terms": {
"location": [
"germany"
]
}
},
{
"bool": {
"must": [
{
"terms": {
"valueType": [
"integer"
]
}
}
]
}
}
]
}
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d4597d15-8785-4e97-9c3f-8be9aacddf9b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Must clauses are queries that must return a document. In the first query,
any document returned MUST have a location of Germany. The valueType should
clause is optional and actually pointless as a filter since it does not
contribute to scoring.

Can you explain what your query should be doing in terms of boolean logic?

--
Ivan

On Sat, Nov 1, 2014 at 4:39 PM, kazoompa rhaeri@p3g.org wrote:

Hi,

Below is my data and the two queries that I tested, first one failing and
the latter working. I start to believe that if one wants to combine several
SHOULD and MUST filters, the outer one must always be SHOULD. Is this a
correct assumption? In our application, we have much more complex situation
with several filters within each MUST and SHOULD. And lastly, where should
place a MUST_NOT in this case?

Many thanks.

Here is my data:

_index,_type,_id,_score,_source.id,_source.type,_source.valueType,_source.sentence,_source.location
"test","var","0","1","0","study","text","Lorem text is jumbled","spain"
"test","var","1","1","1","study","text","bla bla bla","spain"
"test","var","2","1","2","schema","decimal","ipsum","germany"
"test","var","3","1","3","study","integer","lorem","france"

Here is my FAILING query:

{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": {
"terms": {
"location": [
"germany"
]
}
},
"should": {
"terms": {
"valueType": [
"integer"
]
}
}
}
}
}
}
}

Here is my WORKING query returning IDs 2 and 3:

{
"query": {
"bool": {
"should": [
{
"terms": {
"location": [
"germany"
]
}
},
{
"bool": {
"must": [
{
"terms": {
"valueType": [
"integer"
]
}
}
]
}
}
]
}
}
}

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d4597d15-8785-4e97-9c3f-8be9aacddf9b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d4597d15-8785-4e97-9c3f-8be9aacddf9b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCgojLxpzSrYQyW1%3DfeaF_TJdkx4dqgaxq0_sijvq6dvw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Thanks Ivan,

We would like to create complex queries explained in this
page: Elasticsearch Platform — Find real-time answers at scale | Elastic.
I have to admit I don't see why anybody would like to put MUSTs and SHOULDs
at same level. After further analysis, it seems that if I like to do
something like:

(For this example conside A, B and, C as terms filters and 'in' implying
their possible values:)

A in [a1, a2,...] OR B in [b1, b2,...] AND C in [c1, c2. ...] // order is
important

I have to implement my bool filter as:

{
"bool": {
"must": [
{
"bool": {
"should": [
{
"terms": {
"A": [
"a1",
"a2"
]
}
},
{
"terms": {
"B": [
"b1",
"b2"
]
}
}
]
},
"terms": {
"C": [
"c1",
"c2"
]
}
}
]
}
}

It's sort of a Polish notation for queries :wink:

Cheers.

On Monday, November 3, 2014 5:02:15 PM UTC-5, Ivan Brusic wrote:

Must clauses are queries that must return a document. In the first query,
any document returned MUST have a location of Germany. The valueType should
clause is optional and actually pointless as a filter since it does not
contribute to scoring.

Can you explain what your query should be doing in terms of boolean logic?

--
Ivan

On Sat, Nov 1, 2014 at 4:39 PM, kazoompa <rha...@p3g.org <javascript:>>
wrote:

Hi,

Below is my data and the two queries that I tested, first one failing and
the latter working. I start to believe that if one wants to combine several
SHOULD and MUST filters, the outer one must always be SHOULD. Is this a
correct assumption? In our application, we have much more complex situation
with several filters within each MUST and SHOULD. And lastly, where should
place a MUST_NOT in this case?

Many thanks.

Here is my data:

_index,_type,_id,_score,_source.id,_source.type,_source.valueType,_source.sentence,_source.location
"test","var","0","1","0","study","text","Lorem text is jumbled","spain"
"test","var","1","1","1","study","text","bla bla bla","spain"
"test","var","2","1","2","schema","decimal","ipsum","germany"
"test","var","3","1","3","study","integer","lorem","france"

Here is my FAILING query:

{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": {
"terms": {
"location": [
"germany"
]
}
},
"should": {
"terms": {
"valueType": [
"integer"
]
}
}
}
}
}
}
}

Here is my WORKING query returning IDs 2 and 3:

{
"query": {
"bool": {
"should": [
{
"terms": {
"location": [
"germany"
]
}
},
{
"bool": {
"must": [
{
"terms": {
"valueType": [
"integer"
]
}
}
]
}
}
]
}
}
}

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d4597d15-8785-4e97-9c3f-8be9aacddf9b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d4597d15-8785-4e97-9c3f-8be9aacddf9b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a33a6974-4a95-4632-9c56-bea3d19ce7f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Should clauses at the same time as must clauses are only important during
queries (not filters) since they contribute to the scoring for a document.
The should clauses will improve the score for the documents that match.

--
Ivan

On Mon, Nov 3, 2014 at 5:51 PM, kazoompa rhaeri@p3g.org wrote:

Thanks Ivan,

We would like to create complex queries explained in this page:
Elasticsearch Platform — Find real-time answers at scale | Elastic.
I have to admit I don't see why anybody would like to put MUSTs and SHOULDs
at same level. After further analysis, it seems that if I like to do
something like:

(For this example conside A, B and, C as terms filters and 'in' implying
their possible values:)

A in [a1, a2,...] OR B in [b1, b2,...] AND C in [c1, c2. ...] // order
is important

I have to implement my bool filter as:

{
"bool": {
"must": [
{
"bool": {
"should": [
{
"terms": {
"A": [
"a1",
"a2"
]
}
},
{
"terms": {
"B": [
"b1",
"b2"
]
}
}
]
},
"terms": {
"C": [
"c1",
"c2"
]
}
}
]
}
}

It's sort of a Polish notation for queries :wink:

Cheers.

On Monday, November 3, 2014 5:02:15 PM UTC-5, Ivan Brusic wrote:

Must clauses are queries that must return a document. In the first query,
any document returned MUST have a location of Germany. The valueType should
clause is optional and actually pointless as a filter since it does not
contribute to scoring.

Can you explain what your query should be doing in terms of boolean logic?

--
Ivan

On Sat, Nov 1, 2014 at 4:39 PM, kazoompa rha...@p3g.org wrote:

Hi,

Below is my data and the two queries that I tested, first one failing
and the latter working. I start to believe that if one wants to combine
several SHOULD and MUST filters, the outer one must always be SHOULD. Is
this a correct assumption? In our application, we have much more complex
situation with several filters within each MUST and SHOULD. And lastly,
where should place a MUST_NOT in this case?

Many thanks.

Here is my data:

_index,_type,_id,_score,_source.id,_source.type,_source.valueType,_source.sentence,_source.location
"test","var","0","1","0","study","text","Lorem text is jumbled","spain"
"test","var","1","1","1","study","text","bla bla bla","spain"
"test","var","2","1","2","schema","decimal","ipsum","germany"
"test","var","3","1","3","study","integer","lorem","france"

Here is my FAILING query:

{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": {
"terms": {
"location": [
"germany"
]
}
},
"should": {
"terms": {
"valueType": [
"integer"
]
}
}
}
}
}
}
}

Here is my WORKING query returning IDs 2 and 3:

{
"query": {
"bool": {
"should": [
{
"terms": {
"location": [
"germany"
]
}
},
{
"bool": {
"must": [
{
"terms": {
"valueType": [
"integer"
]
}
}
]
}
}
]
}
}
}

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/d4597d15-8785-4e97-9c3f-8be9aacddf9b%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d4597d15-8785-4e97-9c3f-8be9aacddf9b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a33a6974-4a95-4632-9c56-bea3d19ce7f0%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a33a6974-4a95-4632-9c56-bea3d19ce7f0%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBRFfcTDA1Q3S8JKpkECayGoL-qtr2Z9EDrbku_%3D_bQMQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Thanks Ivan,

We finally opted for building our queries (thru a UI query builder) in a
nested fashion as dscribed above, it seems to serve our need.

Cheers for the info though.

On Tuesday, November 4, 2014 11:55:27 AM UTC-5, Ivan Brusic wrote:

Should clauses at the same time as must clauses are only important during
queries (not filters) since they contribute to the scoring for a document.
The should clauses will improve the score for the documents that match.

--
Ivan

On Mon, Nov 3, 2014 at 5:51 PM, kazoompa <rha...@p3g.org <javascript:>>
wrote:

Thanks Ivan,

We would like to create complex queries explained in this page:
Elasticsearch Platform — Find real-time answers at scale | Elastic.
I have to admit I don't see why anybody would like to put MUSTs and SHOULDs
at same level. After further analysis, it seems that if I like to do
something like:

(For this example conside A, B and, C as terms filters and 'in' implying
their possible values:)

A in [a1, a2,...] OR B in [b1, b2,...] AND C in [c1, c2. ...] // order
is important

I have to implement my bool filter as:

{
"bool": {
"must": [
{
"bool": {
"should": [
{
"terms": {
"A": [
"a1",
"a2"
]
}
},
{
"terms": {
"B": [
"b1",
"b2"
]
}
}
]
},
"terms": {
"C": [
"c1",
"c2"
]
}
}
]
}
}

It's sort of a Polish notation for queries :wink:

Cheers.

On Monday, November 3, 2014 5:02:15 PM UTC-5, Ivan Brusic wrote:

Must clauses are queries that must return a document. In the first
query, any document returned MUST have a location of Germany. The valueType
should clause is optional and actually pointless as a filter since it does
not contribute to scoring.

Can you explain what your query should be doing in terms of boolean
logic?

--
Ivan

On Sat, Nov 1, 2014 at 4:39 PM, kazoompa rha...@p3g.org wrote:

Hi,

Below is my data and the two queries that I tested, first one failing
and the latter working. I start to believe that if one wants to combine
several SHOULD and MUST filters, the outer one must always be SHOULD. Is
this a correct assumption? In our application, we have much more complex
situation with several filters within each MUST and SHOULD. And lastly,
where should place a MUST_NOT in this case?

Many thanks.

Here is my data:

_index,_type,_id,_score,_source.id,_source.type,_source.valueType,_source.sentence,_source.location
"test","var","0","1","0","study","text","Lorem text is jumbled","spain"
"test","var","1","1","1","study","text","bla bla bla","spain"
"test","var","2","1","2","schema","decimal","ipsum","germany"
"test","var","3","1","3","study","integer","lorem","france"

Here is my FAILING query:

{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": {
"terms": {
"location": [
"germany"
]
}
},
"should": {
"terms": {
"valueType": [
"integer"
]
}
}
}
}
}
}
}

Here is my WORKING query returning IDs 2 and 3:

{
"query": {
"bool": {
"should": [
{
"terms": {
"location": [
"germany"
]
}
},
{
"bool": {
"must": [
{
"terms": {
"valueType": [
"integer"
]
}
}
]
}
}
]
}
}
}

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/d4597d15-8785-4e97-9c3f-8be9aacddf9b%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d4597d15-8785-4e97-9c3f-8be9aacddf9b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a33a6974-4a95-4632-9c56-bea3d19ce7f0%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a33a6974-4a95-4632-9c56-bea3d19ce7f0%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0be6e810-9b08-47b0-b071-93b601c48626%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.