# An issue with must\_not clause
**URL:** https://discuss.elastic.co/t/an-issue-with-must-not-clause/7056
**Category:** Elasticsearch
**Created:** [March 19, 2012, 2:26pm UTC](https://discuss.elastic.co/t/an-issue-with-must-not-clause/7056 "2012-03-19T14:26:57Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@Jan\_Palko](https://discuss.elastic.co/u/Jan_Palko)
#### Post date: [March 19, 2012, 2:26pm UTC](https://discuss.elastic.co/t/an-issue-with-must-not-clause/7056/1 "2012-03-19T14:26:57Z")
Hi,
We are using elastisearch for sports events which requires geoblocking due
to rights. Below is final filter from Java API. I removed parts which are
not needed for this issue.
{
"filter" : {
"and" : {
"filters" : [ {
"query" : {
"bool" : {
"should" : [ {
"bool" : {
"must" : [ {
"term" : {
"blockByDefault" : true
}
}, {
"term" : {
"allowedCountryCodes" : "it"
}
} ]
}
}, {
"bool" : {
"must" : {
"term" : {
"blockByDefault" : false
}
},
"must\_not" : {
"term" : {
"blockedCountryCodes" : "it"
}
}
}
} ],
"minimum\_number\_should\_match" : 1
}
}
} ]
}
}
}
When _blockByDefault_ is set to _true_, then it should behave as _whitelist_ (allow
only countries defined in allowedCountryCodes), otherwise _blacklist_ (block
only countries defined in blockedCountryCodes).
Here is the mapping:
{
"event" : {
"properties" : {
"blockByDefault" : {"type" : "boolean", "store" : "yes", "index" :
"not\_analyzed"},
"allowedCountryCodes" : {"type" : "string", "store" : "yes", "index"
: "not\_analyzed"},
"blockedCountryCodes" : {"type" : "string", "store" : "yes", "index"
: "not\_analyzed"}
}
}
}
Here is test data:
{"blockByDefault" : true, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
: ["it", "sk"]}
{"blockByDefault" : true, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
: ["it", "sk"]}
{"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
: ["it", "sk"]}
{"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
: ["it", "sk"]}
{"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
: ["it", "sk"]}
Expected number of results for the following countries are as follows:
- gb 5
- de 3
- sk 2
- it 0 - this is problem, as it returns 3 results (the ones with
blockByDefault set to false) instead of 0.
Everything works except the last case, when all events should be blocked
from Italy, but it doesn't work here.
I've tried to rewrite it in other ways, but still the same results. I've
also tried to filter the events where there is Italy in blockedCountryCodes
and it worked, but then when I added blockByDefault "switch", it returned
bad results. What am I doing wrong?
Thanks
Jan
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [March 19, 2012, 5:29pm UTC](https://discuss.elastic.co/t/an-issue-with-must-not-clause/7056/2 "2012-03-19T17:29:33Z")
You are using default analyzer.
"It" is a common english word which is ignored.
Applying a keyword mapping (with lowercase filter) should solve your issue.
HTH
David 😉
Twitter : @dadoonet / @elasticsearchfr
Le 19 mars 2012 à 15:26, Ján Paľko [jan.palko@gmail.com](mailto:jan.palko@gmail.com) a écrit :
> Hi,
>
> We are using elastisearch for sports events which requires geoblocking due to rights. Below is final filter from Java API. I removed parts which are not needed for this issue.
>
> {
> "filter" : {
> "and" : {
> "filters" : [ {
> "query" : {
> "bool" : {
> "should" : [ {
> "bool" : {
> "must" : [ {
> "term" : {
> "blockByDefault" : true
> }
> }, {
> "term" : {
> "allowedCountryCodes" : "it"
> }
> } ]
> }
> }, {
> "bool" : {
> "must" : {
> "term" : {
> "blockByDefault" : false
> }
> },
> "must\_not" : {
> "term" : {
> "blockedCountryCodes" : "it"
> }
> }
> }
> } ],
> "minimum\_number\_should\_match" : 1
> }
> }
> } ]
> }
> }
> }
>
> When blockByDefault is set to true, then it should behave as whitelist (allow only countries defined in allowedCountryCodes), otherwise blacklist (block only countries defined in blockedCountryCodes).
>
> Here is the mapping:
> {
> "event" : {
> "properties" : {
> "blockByDefault" : {"type" : "boolean", "store" : "yes", "index" : "not\_analyzed"},
> "allowedCountryCodes" : {"type" : "string", "store" : "yes", "index" : "not\_analyzed"},
> "blockedCountryCodes" : {"type" : "string", "store" : "yes", "index" : "not\_analyzed"}
> }
> }
> }
>
> Here is test data:
> {"blockByDefault" : true, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes" : ["it", "sk"]}
> {"blockByDefault" : true, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes" : ["it", "sk"]}
> {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes" : ["it", "sk"]}
> {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes" : ["it", "sk"]}
> {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes" : ["it", "sk"]}
>
> Expected number of results for the following countries are as follows:
> gb 5
> de 3
> sk 2
> it 0 - this is problem, as it returns 3 results (the ones with blockByDefault set to false) instead of 0.
> Everything works except the last case, when all events should be blocked from Italy, but it doesn't work here.
>
> I've tried to rewrite it in other ways, but still the same results. I've also tried to filter the events where there is Italy in blockedCountryCodes and it worked, but then when I added blockByDefault "switch", it returned bad results. What am I doing wrong?
>
> Thanks
> Jan
---
### Author:  [@Jan\_Palko](https://discuss.elastic.co/u/Jan_Palko)
#### Post date: [March 20, 2012, 7:33am UTC](https://discuss.elastic.co/t/an-issue-with-must-not-clause/7056/3 "2012-03-20T07:33:38Z")
Many Thanks 🙂 It works. But it's strange that I had tried to set index to
not\_analyzed and it didn't work and now does.
Jan
On Monday, March 19, 2012 6:29:33 PM UTC+1, David Pilato wrote:
> You are using default analyzer.
> "It" is a common english word which is ignored.
>
> Applying a keyword mapping (with lowercase filter) should solve your issue.
>
> HTH
> David 😉
> Twitter : @dadoonet / @elasticsearchfr
>
> Le 19 mars 2012 à 15:26, Ján Paľko a écrit :
>
> Hi,
>
> We are using elastisearch for sports events which requires geoblocking due
> to rights. Below is final filter from Java API. I removed parts which are
> not needed for this issue.
>
> {
> "filter" : {
> "and" : {
> "filters" : [ {
> "query" : {
> "bool" : {
> "should" : [ {
> "bool" : {
> "must" : [ {
> "term" : {
> "blockByDefault" : true
> }
> }, {
> "term" : {
> "allowedCountryCodes" : "it"
> }
> } ]
> }
> }, {
> "bool" : {
> "must" : {
> "term" : {
> "blockByDefault" : false
> }
> },
> "must\_not" : {
> "term" : {
> "blockedCountryCodes" : "it"
> }
> }
> }
> } ],
> "minimum\_number\_should\_match" : 1
> }
> }
> } ]
> }
> }
> }
>
> When _blockByDefault_ is set to _true_, then it should behave as \*
> whitelist\* (allow only countries defined in allowedCountryCodes),
> otherwise _blacklist_ (block only countries defined in
> blockedCountryCodes).
>
> Here is the mapping:
>
> {
> "event" : {
> "properties" : {
> "blockByDefault" : {"type" : "boolean", "store" : "yes", "index" :
> "not\_analyzed"},
> "allowedCountryCodes" : {"type" : "string", "store" : "yes", "index"
> : "not\_analyzed"},
> "blockedCountryCodes" : {"type" : "string", "store" : "yes", "index"
> : "not\_analyzed"}
> }
> }
> }
>
> Here is test data:
>
> {"blockByDefault" : true, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
> : ["it", "sk"]}
>
> {"blockByDefault" : true, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
> : ["it", "sk"]}
> {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
> : ["it", "sk"]}
> {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
> : ["it", "sk"]}
> {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
> : ["it", "sk"]}
>
> Expected number of results for the following countries are as follows:
>
> - gb 5
> - de 3
> - sk 2
> - it 0 - this is problem, as it returns 3 results (the ones with
> blockByDefault set to false) instead of 0.
>
> Everything works except the last case, when all events should be blocked
> from Italy, but it doesn't work here.
>
> I've tried to rewrite it in other ways, but still the same results. I've
> also tried to filter the events where there is Italy in blockedCountryCodes
> and it worked, but then when I added blockByDefault "switch", it returned
> bad results. What am I doing wrong?
>
> Thanks
> Jan
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [March 20, 2012, 10:58am UTC](https://discuss.elastic.co/t/an-issue-with-must-not-clause/7056/4 "2012-03-20T10:58:36Z")
One more thing, you are using a filter, and then wrap it with a query, its
a shame, you should use filters all the way in this case (bool filter, with
term filters for example).
2012/3/20 Ján Paľko [jan.palko@gmail.com](mailto:jan.palko@gmail.com)
> Many Thanks 🙂 It works. But it's strange that I had tried to set index to
> not\_analyzed and it didn't work and now does.
>
> Jan
>
> On Monday, March 19, 2012 6:29:33 PM UTC+1, David Pilato wrote:
>
> > You are using default analyzer.
> > "It" is a common english word which is ignored.
> >
> > Applying a keyword mapping (with lowercase filter) should solve your
> > issue.
> >
> > HTH
> > David 😉
> > Twitter : @dadoonet / @elasticsearchfr
> >
> > Le 19 mars 2012 à 15:26, Ján Paľko a écrit :
> >
> > Hi,
> >
> > We are using elastisearch for sports events which requires geoblocking
> > due to rights. Below is final filter from Java API. I removed parts which
> > are not needed for this issue.
> >
> > {
> > "filter" : {
> > "and" : {
> > "filters" : [ {
> > "query" : {
> > "bool" : {
> > "should" : [ {
> > "bool" : {
> > "must" : [ {
> > "term" : {
> > "blockByDefault" : true
> > }
> > }, {
> > "term" : {
> > "allowedCountryCodes" : "it"
> > }
> > } ]
> > }
> > }, {
> > "bool" : {
> > "must" : {
> > "term" : {
> > "blockByDefault" : false
> > }
> > },
> > "must\_not" : {
> > "term" : {
> > "blockedCountryCodes" : "it"
> > }
> > }
> > }
> > } ],
> > "minimum\_number\_should\_match" : 1
> > }
> > }
> > } ]
> > }
> > }
> > }
> >
> > When _blockByDefault_ is set to _true_, then it should behave as \*
> > whitelist\* (allow only countries defined in allowedCountryCodes),
> > otherwise _blacklist_ (block only countries defined in
> > blockedCountryCodes).
> >
> > Here is the mapping:
> >
> > {
> > "event" : {
> > "properties" : {
> > "blockByDefault" : {"type" : "boolean", "store" : "yes", "index" :
> > "not\_analyzed"},
> > "allowedCountryCodes" : {"type" : "string", "store" : "yes",
> > "index" : "not\_analyzed"},
> > "blockedCountryCodes" : {"type" : "string", "store" : "yes",
> > "index" : "not\_analyzed"}
> > }
> > }
> > }
> >
> > Here is test data:
> >
> > {"blockByDefault" : true, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
> > : ["it", "sk"]}
> >
> > {"blockByDefault" : true, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
> > : ["it", "sk"]}
> > {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
> > : ["it", "sk"]}
> > {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
> > : ["it", "sk"]}
> > {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"], "blockedCountryCodes"
> > : ["it", "sk"]}
> >
> > Expected number of results for the following countries are as follows:
> >
> > - gb 5
> > - de 3
> > - sk 2
> > - it 0 - this is problem, as it returns 3 results (the ones with
> > blockByDefault set to false) instead of 0.
> >
> > Everything works except the last case, when all events should be blocked
> > from Italy, but it doesn't work here.
> >
> > I've tried to rewrite it in other ways, but still the same results. I've
> > also tried to filter the events where there is Italy in blockedCountryCodes
> > and it worked, but then when I added blockByDefault "switch", it returned
> > bad results. What am I doing wrong?
> >
> > Thanks
> > Jan
---
### Author:  [@Jan\_Palko](https://discuss.elastic.co/u/Jan_Palko)
#### Post date: [March 20, 2012, 11:53am UTC](https://discuss.elastic.co/t/an-issue-with-must-not-clause/7056/5 "2012-03-20T11:53:57Z")
Is there any impact on performance? When I use just a query, it's
slower as it does scoring. Is it the same when I use it inside filter?
Thanks
On Tue, Mar 20, 2012 at 11:58, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> One more thing, you are using a filter, and then wrap it with a query, its a
> shame, you should use filters all the way in this case (bool filter, with
> term filters for example).
>
> 2012/3/20 Ján Paľko [jan.palko@gmail.com](mailto:jan.palko@gmail.com)
>
> > Many Thanks 🙂 It works. But it's strange that I had tried to set index to
> > not\_analyzed and it didn't work and now does.
> >
> > Jan
> >
> > On Monday, March 19, 2012 6:29:33 PM UTC+1, David Pilato wrote:
> >
> > > You are using default analyzer.
> > > "It" is a common english word which is ignored.
> > >
> > > Applying a keyword mapping (with lowercase filter) should solve your
> > > issue.
> > >
> > > HTH
> > > David 😉
> > > Twitter : @dadoonet / @elasticsearchfr
> > >
> > > Le 19 mars 2012 à 15:26, Ján Paľko a écrit :
> > >
> > > Hi,
> > >
> > > We are using elastisearch for sports events which requires geoblocking
> > > due to rights. Below is final filter from Java API. I removed parts which
> > > are not needed for this issue.
> > >
> > > {
> > > "filter" : {
> > > "and" : {
> > > "filters" : [ {
> > > "query" : {
> > > "bool" : {
> > > "should" : [ {
> > > "bool" : {
> > > "must" : [ {
> > > "term" : {
> > > "blockByDefault" : true
> > > }
> > > }, {
> > > "term" : {
> > > "allowedCountryCodes" : "it"
> > > }
> > > } ]
> > > }
> > > }, {
> > > "bool" : {
> > > "must" : {
> > > "term" : {
> > > "blockByDefault" : false
> > > }
> > > },
> > > "must\_not" : {
> > > "term" : {
> > > "blockedCountryCodes" : "it"
> > > }
> > > }
> > > }
> > > } ],
> > > "minimum\_number\_should\_match" : 1
> > > }
> > > }
> > > } ]
> > > }
> > > }
> > > }
> > >
> > > When blockByDefault is set to true, then it should behave
> > > as whitelist (allow only countries defined in allowedCountryCodes),
> > > otherwise blacklist (block only countries defined in blockedCountryCodes).
> > >
> > > Here is the mapping:
> > >
> > > {
> > > "event" : {
> > > "properties" : {
> > > "blockByDefault" : {"type" : "boolean", "store" : "yes", "index" :
> > > "not\_analyzed"},
> > > "allowedCountryCodes" : {"type" : "string", "store" : "yes",
> > > "index" : "not\_analyzed"},
> > > "blockedCountryCodes" : {"type" : "string", "store" : "yes",
> > > "index" : "not\_analyzed"}
> > > }
> > > }
> > > }
> > >
> > > Here is test data:
> > >
> > > {"blockByDefault" : true, "allowedCountryCodes" : ["gb", "sk"],
> > > "blockedCountryCodes" : ["it", "sk"]}
> > >
> > > {"blockByDefault" : true, "allowedCountryCodes" : ["gb", "sk"],
> > > "blockedCountryCodes" : ["it", "sk"]}
> > > {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"],
> > > "blockedCountryCodes" : ["it", "sk"]}
> > > {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"],
> > > "blockedCountryCodes" : ["it", "sk"]}
> > > {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"],
> > > "blockedCountryCodes" : ["it", "sk"]}
> > >
> > > Expected number of results for the following countries are as follows:
> > >
> > > gb 5
> > > de 3
> > > sk 2
> > > it 0 - this is problem, as it returns 3 results (the ones with
> > > blockByDefault set to false) instead of 0.
> > >
> > > Everything works except the last case, when all events should be blocked
> > > from Italy, but it doesn't work here.
> > >
> > > I've tried to rewrite it in other ways, but still the same results. I've
> > > also tried to filter the events where there is Italy in blockedCountryCodes
> > > and it worked, but then when I added blockByDefault "switch", it returned
> > > bad results. What am I doing wrong?
> > >
> > > Thanks
> > > Jan
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [March 20, 2012, 8:31pm UTC](https://discuss.elastic.co/t/an-issue-with-must-not-clause/7056/6 "2012-03-20T20:31:31Z")
If you transform the internal query in the filter to be based on filters,
then you will gain the benefits of filter caching. Scoring will not be
computed in any case.
2012/3/20 Ján Paľko [jan.palko@gmail.com](mailto:jan.palko@gmail.com)
> Is there any impact on performance? When I use just a query, it's
> slower as it does scoring. Is it the same when I use it inside filter?
>
> Thanks
>
> On Tue, Mar 20, 2012 at 11:58, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
>
> > One more thing, you are using a filter, and then wrap it with a query,
> > its a
> > shame, you should use filters all the way in this case (bool filter, with
> > term filters for example).
> >
> > 2012/3/20 Ján Paľko [jan.palko@gmail.com](mailto:jan.palko@gmail.com)
> >
> > > Many Thanks 🙂 It works. But it's strange that I had tried to set index
> > > to
> > > not\_analyzed and it didn't work and now does.
> > >
> > > Jan
> > >
> > > On Monday, March 19, 2012 6:29:33 PM UTC+1, David Pilato wrote:
> > >
> > > > You are using default analyzer.
> > > > "It" is a common english word which is ignored.
> > > >
> > > > Applying a keyword mapping (with lowercase filter) should solve your
> > > > issue.
> > > >
> > > > HTH
> > > > David 😉
> > > > Twitter : @dadoonet / @elasticsearchfr
> > > >
> > > > Le 19 mars 2012 à 15:26, Ján Paľko a écrit :
> > > >
> > > > Hi,
> > > >
> > > > We are using elastisearch for sports events which requires geoblocking
> > > > due to rights. Below is final filter from Java API. I removed parts
> > > > which
> > > > are not needed for this issue.
> > > >
> > > > {
> > > > "filter" : {
> > > > "and" : {
> > > > "filters" : [ {
> > > > "query" : {
> > > > "bool" : {
> > > > "should" : [ {
> > > > "bool" : {
> > > > "must" : [ {
> > > > "term" : {
> > > > "blockByDefault" : true
> > > > }
> > > > }, {
> > > > "term" : {
> > > > "allowedCountryCodes" : "it"
> > > > }
> > > > } ]
> > > > }
> > > > }, {
> > > > "bool" : {
> > > > "must" : {
> > > > "term" : {
> > > > "blockByDefault" : false
> > > > }
> > > > },
> > > > "must\_not" : {
> > > > "term" : {
> > > > "blockedCountryCodes" : "it"
> > > > }
> > > > }
> > > > }
> > > > } ],
> > > > "minimum\_number\_should\_match" : 1
> > > > }
> > > > }
> > > > } ]
> > > > }
> > > > }
> > > > }
> > > >
> > > > When blockByDefault is set to true, then it should behave
> > > > as whitelist (allow only countries defined in allowedCountryCodes),
> > > > otherwise blacklist (block only countries defined in
> > > > blockedCountryCodes).
> > > >
> > > > Here is the mapping:
> > > >
> > > > {
> > > > "event" : {
> > > > "properties" : {
> > > > "blockByDefault" : {"type" : "boolean", "store" : "yes", "index"
> > > > :
> > > > "not\_analyzed"},
> > > > "allowedCountryCodes" : {"type" : "string", "store" : "yes",
> > > > "index" : "not\_analyzed"},
> > > > "blockedCountryCodes" : {"type" : "string", "store" : "yes",
> > > > "index" : "not\_analyzed"}
> > > > }
> > > > }
> > > > }
> > > >
> > > > Here is test data:
> > > >
> > > > {"blockByDefault" : true, "allowedCountryCodes" : ["gb", "sk"],
> > > > "blockedCountryCodes" : ["it", "sk"]}
> > > >
> > > > {"blockByDefault" : true, "allowedCountryCodes" : ["gb", "sk"],
> > > > "blockedCountryCodes" : ["it", "sk"]}
> > > > {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"],
> > > > "blockedCountryCodes" : ["it", "sk"]}
> > > > {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"],
> > > > "blockedCountryCodes" : ["it", "sk"]}
> > > > {"blockByDefault" : false, "allowedCountryCodes" : ["gb", "sk"],
> > > > "blockedCountryCodes" : ["it", "sk"]}
> > > >
> > > > Expected number of results for the following countries are as follows:
> > > >
> > > > gb 5
> > > > de 3
> > > > sk 2
> > > > it 0 - this is problem, as it returns 3 results (the ones with
> > > > blockByDefault set to false) instead of 0.
> > > >
> > > > Everything works except the last case, when all events should be
> > > > blocked
> > > > from Italy, but it doesn't work here.
> > > >
> > > > I've tried to rewrite it in other ways, but still the same results.
> > > > I've
> > > > also tried to filter the events where there is Italy in
> > > > blockedCountryCodes
> > > > and it worked, but then when I added blockByDefault "switch", it
> > > > returned
> > > > bad results. What am I doing wrong?
> > > >
> > > > Thanks
> > > > Jan
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:35am UTC](https://discuss.elastic.co/t/an-issue-with-must-not-clause/7056/7 "2017-07-06T03:35:23Z")