# Inexplicable behavior using 'terms' filter **URL:** \ **Category:** Elasticsearch\ **Created:** [April 24, 2014, 3:51pm UTC](https://discuss.elastic.co/t/inexplicable-behavior-using-terms-filter/17179 "2014-04-24T15:51:18Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Luca\_Puccini](https://avatars.discourse-cdn.com/v4/letter/l/9dc877/32.png) [@Luca\_Puccini](https://discuss.elastic.co/u/Luca_Puccini)\ **Post date:** [April 24, 2014, 3:51pm UTC](https://discuss.elastic.co/t/inexplicable-behavior-using-terms-filter/17179/1 "2014-04-24T15:51:18Z")
Hello everyone, I'm pretty new to ES, so I hope I won't be writing any blasfemity. In this case please forgive me. I have an ES index { "settings": { "analysis": { "analyzer": { "de\_analyzer": { "type": "snowball", "language": "German" } } } } } with an ES map uselessF The data, retrieved by a river from a mongodb instance, are in the form: { "someInfo1" : { ... } "someInfo1" : { ... } "someInfo1" : { ... } "importantInfo" : { "field1": true, "field2": 2123848121, ... "magnetId": "stringId1", "type": "stringType" } } all data are correctly indexed by the river, but when I execute a query with the _terms_ filter in it I got no results, even though there are some in the index, of this I'm sure (please notice in the query below that _importantInfo.importantFlag_ is not present in the map). { "query": { "filtered": { "query": { "multi\_match": { "fields": [ "importantInfo.toBeSearched1^2", "importantInfo.toBeSearched2" ], "query": "peer" } }, "filter": { "and": [ { "term": { "importantInfo.importantField": "stringField" } }, { "term": { "importantInfo.importantFlag": true } }, _{ "terms" : { "importantInfo.importantId": ["stringId1", "stringId2"] } }_ ] } } }, "size": 10, "from": 0, "fields": [ "importantInfo.toBeSearched1", "importantInfo.toBeSearched2", "importantInfo.importantId" ] } If I use the following query instead, I got some results that respect all the filters { "query": { "filtered": { "query": { "multi\_match": { "fields": [ "importantInfo.toBeSearched1^2", "importantInfo.toBeSearched2" ], "query": "peer" } }, "filter": { "and": [ { "term": { "importantInfo.importantField": "stringField" } }, { "term": { "importantInfo.importantFlag": true } } ] } } }, "size": 10, "from": 0, "fields": [ "importantInfo.toBeSearched1", "importantInfo.toBeSearched2", "importantInfo.importantId" ] } Also, when I use terms, for example, on importantField instead of importantId I have correct results as well. { "query": { "filtered": { "query": { "multi\_match": { "fields": [ "importantInfo.toBeSearched1^2", "importantInfo.toBeSearched2" ], "query": "peer" } }, "filter": { "and": [ \*{ "terms" : { "importantInfo.importantField": [ "\*_stringField1", "_ _stringField2" ] } }_, { "term": { "importantInfo.importantFlag": true } } ] } } }, "size": 10, "from": 0, "fields": [ "importantInfo.toBeSearched1", "importantInfo.toBeSearched2", "importantInfo.importantId" ] } I'm trying hard to understand what could possibly go wrong with that filter. The field is mapped exactly as the others, but it cannot be used to filter results. I'm setting up a script to simulate this behavior and I will post it here soon as it'll be ready, but in the meantime maybe someone can spot some error I've made or option I've missed. thanks in advance L -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/eb92d62d-db0c-4ab4-9aa3-3ae03de2b4b0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/eb92d62d-db0c-4ab4-9aa3-3ae03de2b4b0%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Luca\_Puccini](https://avatars.discourse-cdn.com/v4/letter/l/9dc877/32.png) [@Luca\_Puccini](https://discuss.elastic.co/u/Luca_Puccini)\ **Post date:** [April 24, 2014, 3:53pm UTC](https://discuss.elastic.co/t/inexplicable-behavior-using-terms-filter/17179/2 "2014-04-24T15:53:42Z")
Of course the map is wrong. here's the right one ES map: { "mapName": { "\_all" : {"enabled" : false}, "properties": { "uselessField1": { "type": "boolean", "index": "no" }, "uselessField2": { "type": "long", "index": "no" }, ... "toBeSearched1": { "type": "string", "index": "de\_analyzer" }, "toBeSearched2": { "type": "string", "index": "de\_analyzer" }, "importantId": { "type": "string", "index": "not\_analyzed" }, "importantField": { "type": "string", "index": "not\_analyzed" } } } } -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d34e9092-e6f4-4a9f-b545-662154ea80d6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d34e9092-e6f4-4a9f-b545-662154ea80d6%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [April 24, 2014, 4:42pm UTC](https://discuss.elastic.co/t/inexplicable-behavior-using-terms-filter/17179/3 "2014-04-24T16:42:00Z")
luca, what do the values in 'importantId` look like? Are they identical to the values you pass in? The field is not analyzed though. It would also be helpful if you could show the results that you would expect so we can see what values they have. simon On Thursday, April 24, 2014 5:53:42 PM UTC+2, Luca wrote: > Of course the map is wrong. here's the right one > > ES map: > { > "mapName": { > "\_all" : {"enabled" : false}, > "properties": { > "uselessField1": { > "type": "boolean", > "index": "no" > }, > "uselessField2": { > "type": "long", > "index": "no" > }, > ... > "toBeSearched1": { > "type": "string", > "index": "de\_analyzer" > }, > "toBeSearched2": { > "type": "string", > "index": "de\_analyzer" > }, > "importantId": { > "type": "string", > "index": "not\_analyzed" > }, > "importantField": { > "type": "string", > "index": "not\_analyzed" > } > } > } > } -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Luca\_Puccini](https://avatars.discourse-cdn.com/v4/letter/l/9dc877/32.png) [@Luca\_Puccini](https://discuss.elastic.co/u/Luca_Puccini)\ **Post date:** [April 25, 2014, 2:37pm UTC](https://discuss.elastic.co/t/inexplicable-behavior-using-terms-filter/17179/4 "2014-04-25T14:37:00Z")
Hi Simon, thank you for your answer: the values of importantId field are a finite set of strings. Those values are 100% identical to the ones required in the query. They are not analyzed because, correct me if I'm wrong, here[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html)says that the filter works on _not analyzed_ fields. Just to be clear, suppose that importantId can assume these values: ['apple', 'pear', 'orange', 'apricot', 'banana'] I want the filter to be able to eliminate all the elements that are not ['apple', 'banana']. About the results I'm building an example, and soon as it will be running I'll send it here, hopefully by the end of the day. thanks again! Luca -- Luca Puccini dev @ [mixd.tv](http://mixd.tv) [www.mixd.tv](http://www.mixd.tv) [http://www.crowdengineering.com](http://www.crowdengineering.com) mailto: [luca.puccini@mixd.tv](mailto:luca.puccini@mixd.tv) [luca.puccini@crowdengineering.com](mailto:luca.puccini@crowdengineering.com) Skype: lucap283 On Thu, Apr 24, 2014 at 6:42 PM, simonw [simon.willnauer@elasticsearch.com](mailto:simon.willnauer@elasticsearch.com)wrote: > luca, what do the values in 'importantId` look like? Are they identical to > the values you pass in? The field is not analyzed though. It would also be > helpful if you could show the results that you would expect so we can see > what values they have. > > simon > > On Thursday, April 24, 2014 5:53:42 PM UTC+2, Luca wrote: > > > Of course the map is wrong. here's the right one > > > > ES map: > > { > > "mapName": { > > "\_all" : {"enabled" : false}, > > "properties": { > > "uselessField1": { > > "type": "boolean", > > "index": "no" > > }, > > "uselessField2": { > > "type": "long", > > "index": "no" > > }, > > ... > > "toBeSearched1": { > > "type": "string", > > "index": "de\_analyzer" > > }, > > "toBeSearched2": { > > "type": "string", > > "index": "de\_analyzer" > > }, > > "importantId": { > > "type": "string", > > "index": "not\_analyzed" > > }, > > "importantField": { > > "type": "string", > > "index": "not\_analyzed" > > } > > } > > } > > } > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/\_7lP\_AJrk-c/unsubscribe](https://groups.google.com/d/topic/elasticsearch/_7lP_AJrk-c/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CA%2B0cMOLieuCoqmLousA\_ZK%2B6DbqToB%3DxnM8LL8u95ZVCKKAi1Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CA%2B0cMOLieuCoqmLousA_ZK%2B6DbqToB%3DxnM8LL8u95ZVCKKAi1Q%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Luca\_Puccini](https://avatars.discourse-cdn.com/v4/letter/l/9dc877/32.png) [@Luca\_Puccini](https://discuss.elastic.co/u/Luca_Puccini)\ **Post date:** [April 25, 2014, 3:39pm UTC](https://discuss.elastic.co/t/inexplicable-behavior-using-terms-filter/17179/5 "2014-04-25T15:39:22Z")
Attached you can find a script that basically create an index, a map and add 3 elements to that. the following queries returns respectively 1 (correctly) and 0 results { "query": { "filtered": { "query": { "multi\_match": { "fields": [ "importantInfo.toBeSearched1^2", "importantInfo.toBeSearched2" ], "query": "termA" } }, "filter": { "and": [ { "term": { "importantInfo.fieldNotInMap": true } } ] } } } } { "query": { "filtered": { "query": { "multi\_match": { "fields": [ "importantInfo.toBeSearched1^2", "importantInfo.toBeSearched2" ], "query": "termA" } }, "filter": { "and": [ { "term": { "importantInfo.fieldNotInMap": true } }, { "term" : { "importantInfo.importantField": "typeA" } }, { "terms" : { "importantInfo.importantId": [ "stringId2", "stringId1" ] } } ] } } } } As I told you before this is not 100% my problem. On my system I can actually filter using _terms_ on another field (_importantField_ in the above example) that is exactly of the same type of _importantId_. I haven't been able to reproduce that. BTW if I could understand how to manage terms in this case in order to have the expected result it would be great. thanks Luca On Friday, April 25, 2014 4:37:00 PM UTC+2, Luca wrote: > Hi Simon, thank you for your answer: > > the values of importantId field are a finite set of strings. Those values > are 100% identical to the ones required in the query. They are not analyzed > because, correct me if I'm wrong, here[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html)says that the filter works on _not > analyzed_ fields. > > Just to be clear, suppose that importantId can assume these values: > ['apple', 'pear', 'orange', 'apricot', 'banana'] > I want the filter to be able to eliminate all the elements that are not > ['apple', 'banana']. > > About the results I'm building an example, and soon as it will be running > I'll send it here, hopefully by the end of the day. > > thanks again! > Luca > > On Thu, Apr 24, 2014 at 6:42 PM, simonw \<[simon.willnauer@elasticsearch.com](mailto:simon.willnauer@elasticsearch.com) > > > wrote: > > > luca, what do the values in 'importantId` look like? Are they identical > > to the values you pass in? The field is not analyzed though. It would also > > be helpful if you could show the results that you would expect so we can > > see what values they have. > > > > simon > > > > On Thursday, April 24, 2014 5:53:42 PM UTC+2, Luca wrote: > > > > > Of course the map is wrong. here's the right one > > > > > > ES map: > > > { > > > "mapName": { > > > "\_all" : {"enabled" : false}, > > > "properties": { > > > "uselessField1": { > > > "type": "boolean", > > > "index": "no" > > > }, > > > "uselessField2": { > > > "type": "long", > > > "index": "no" > > > }, > > > ... > > > "toBeSearched1": { > > > "type": "string", > > > "index": "de\_analyzer" > > > }, > > > "toBeSearched2": { > > > "type": "string", > > > "index": "de\_analyzer" > > > }, > > > "importantId": { > > > "type": "string", > > > "index": "not\_analyzed" > > > }, > > > "importantField": { > > > "type": "string", > > > "index": "not\_analyzed" > > > } > > > } > > > } > > > } > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/\_7lP\_AJrk-c/unsubscribe](https://groups.google.com/d/topic/elasticsearch/_7lP_AJrk-c/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/cb904048-7ce7-4018-9792-510e97b851c1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cb904048-7ce7-4018-9792-510e97b851c1%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![javanna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javanna/32/4698_2.png) [@javanna](https://discuss.elastic.co/u/javanna)\ **Post date:** [April 25, 2014, 4:33pm UTC](https://discuss.elastic.co/t/inexplicable-behavior-using-terms-filter/17179/6 "2014-04-25T16:33:15Z")
Hi Luca, I think you should have a look at the analyze and check how your different fields end up in the index. If the second query doesn't return any result it means that the filters you added there don't match any document. What I also noticed is that in your mapping you refer to importantId and so on, but they are under the importantInfo object in your documents. Have a look at your generated mapping too using the get mapping api and see if what you get back is what you submitted or maybe it contains additional fields with default text analysis. Cheers Luca On Friday, April 25, 2014 5:39:22 PM UTC+2, Luca wrote: > Attached you can find a script that basically create an index, a map and > add 3 elements to that. > > the following queries returns respectively 1 (correctly) and 0 results > > { > "query": { > "filtered": { > "query": { > "multi\_match": { > "fields": [ > "importantInfo.toBeSearched1^2", > "importantInfo.toBeSearched2" > ], > "query": "termA" > } > }, > "filter": { > "and": [ > { > "term": { > "importantInfo.fieldNotInMap": true > } > } > ] > } > } > } > } > { > "query": { > "filtered": { > "query": { > "multi\_match": { > "fields": [ > "importantInfo.toBeSearched1^2", > "importantInfo.toBeSearched2" > ], > "query": "termA" > } > }, > "filter": { > "and": [ > { > "term": { > "importantInfo.fieldNotInMap": true > } > }, > { > "term" : { > "importantInfo.importantField": "typeA" > } > }, > { > "terms" : { > "importantInfo.importantId": [ "stringId2", > "stringId1" ] > } > } > ] > } > } > } > } > > As I told you before this is not 100% my problem. On my system I can > actually filter using _terms_ on another field (_importantField_ in the > above example) that is exactly of the same type of _importantId_. I > haven't been able to reproduce that. BTW if I could understand how to > manage terms in this case in order to have the expected result it would be > great. > > thanks > Luca > > On Friday, April 25, 2014 4:37:00 PM UTC+2, Luca wrote: > > > Hi Simon, thank you for your answer: > > > > the values of importantId field are a finite set of strings. Those values > > are 100% identical to the ones required in the query. They are not analyzed > > because, correct me if I'm wrong, here[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html)says that the filter works on _not > > analyzed_ fields. > > > > Just to be clear, suppose that importantId can assume these values: > > ['apple', 'pear', 'orange', 'apricot', 'banana'] > > I want the filter to be able to eliminate all the elements that are not > > ['apple', 'banana']. > > > > About the results I'm building an example, and soon as it will be running > > I'll send it here, hopefully by the end of the day. > > > > thanks again! > > Luca > > > > On Thu, Apr 24, 2014 at 6:42 PM, simonw \< > > [simon.willnauer@elasticsearch.com](mailto:simon.willnauer@elasticsearch.com)\> wrote: > > > > > luca, what do the values in 'importantId` look like? Are they identical > > > to the values you pass in? The field is not analyzed though. It would also > > > be helpful if you could show the results that you would expect so we can > > > see what values they have. > > > > > > simon > > > > > > On Thursday, April 24, 2014 5:53:42 PM UTC+2, Luca wrote: > > > > > > > Of course the map is wrong. here's the right one > > > > > > > > ES map: > > > > { > > > > "mapName": { > > > > "\_all" : {"enabled" : false}, > > > > "properties": { > > > > "uselessField1": { > > > > "type": "boolean", > > > > "index": "no" > > > > }, > > > > "uselessField2": { > > > > "type": "long", > > > > "index": "no" > > > > }, > > > > ... > > > > "toBeSearched1": { > > > > "type": "string", > > > > "index": "de\_analyzer" > > > > }, > > > > "toBeSearched2": { > > > > "type": "string", > > > > "index": "de\_analyzer" > > > > }, > > > > "importantId": { > > > > "type": "string", > > > > "index": "not\_analyzed" > > > > }, > > > > "importantField": { > > > > "type": "string", > > > > "index": "not\_analyzed" > > > > } > > > > } > > > > } > > > > } > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/\_7lP\_AJrk-c/unsubscribe](https://groups.google.com/d/topic/elasticsearch/_7lP_AJrk-c/unsubscribe). > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f719d9a1-3592-4ae2-bed7-54b907a3af16%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f719d9a1-3592-4ae2-bed7-54b907a3af16%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![simonw\_3](https://avatars.discourse-cdn.com/v4/letter/s/ee59a6/32.png) [@simonw\_3](https://discuss.elastic.co/u/simonw_3)\ **Post date:** [April 25, 2014, 6:52pm UTC](https://discuss.elastic.co/t/inexplicable-behavior-using-terms-filter/17179/7 "2014-04-25T18:52:00Z")
as my colleague Luca Cavanna said there is something wrong with your mapping... try this: [mixed\_tv\_mapping · GitHub](https://gist.github.com/s1monw/11299421) this should work with your queries... On Friday, April 25, 2014 6:33:15 PM UTC+2, Luca Cavanna wrote: > Hi Luca, > I think you should have a look at the analyze and check how your different > fields end up in the index. If the second query doesn't return any result > it means that the filters you added there don't match any document. > > What I also noticed is that in your mapping you refer to importantId and > so on, but they are under the importantInfo object in your documents. Have > a look at your generated mapping too using the get mapping api and see if > what you get back is what you submitted or maybe it contains additional > fields with default text analysis. > > Cheers > Luca > > On Friday, April 25, 2014 5:39:22 PM UTC+2, Luca wrote: > > > Attached you can find a script that basically create an index, a map and > > add 3 elements to that. > > > > the following queries returns respectively 1 (correctly) and 0 results > > > > { > > "query": { > > "filtered": { > > "query": { > > "multi\_match": { > > "fields": [ > > "importantInfo.toBeSearched1^2", > > "importantInfo.toBeSearched2" > > ], > > "query": "termA" > > } > > }, > > "filter": { > > "and": [ > > { > > "term": { > > "importantInfo.fieldNotInMap": true > > } > > } > > ] > > } > > } > > } > > } > > { > > "query": { > > "filtered": { > > "query": { > > "multi\_match": { > > "fields": [ > > "importantInfo.toBeSearched1^2", > > "importantInfo.toBeSearched2" > > ], > > "query": "termA" > > } > > }, > > "filter": { > > "and": [ > > { > > "term": { > > "importantInfo.fieldNotInMap": true > > } > > }, > > { > > "term" : { > > "importantInfo.importantField": "typeA" > > } > > }, > > { > > "terms" : { > > "importantInfo.importantId": [ "stringId2", > > "stringId1" ] > > } > > } > > ] > > } > > } > > } > > } > > > > As I told you before this is not 100% my problem. On my system I can > > actually filter using _terms_ on another field (_importantField_ in the > > above example) that is exactly of the same type of _importantId_. I > > haven't been able to reproduce that. BTW if I could understand how to > > manage terms in this case in order to have the expected result it would be > > great. > > > > thanks > > Luca > > > > On Friday, April 25, 2014 4:37:00 PM UTC+2, Luca wrote: > > > > > Hi Simon, thank you for your answer: > > > > > > the values of importantId field are a finite set of strings. Those > > > values are 100% identical to the ones required in the query. They are not > > > analyzed because, correct me if I'm wrong, here[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html)says that the filter works on _not > > > analyzed_ fields. > > > > > > Just to be clear, suppose that importantId can assume these values: > > > ['apple', 'pear', 'orange', 'apricot', 'banana'] > > > I want the filter to be able to eliminate all the elements that are not > > > ['apple', 'banana']. > > > > > > About the results I'm building an example, and soon as it will be > > > running I'll send it here, hopefully by the end of the day. > > > > > > thanks again! > > > Luca > > > > > > On Thu, Apr 24, 2014 at 6:42 PM, simonw \<[simon.w...@elasticsearch.com](mailto:simon.w...@elasticsearch.com)\ > > > > > > > wrote: > > > > > > > luca, what do the values in 'importantId` look like? Are they identical > > > > to the values you pass in? The field is not analyzed though. It would also > > > > be helpful if you could show the results that you would expect so we can > > > > see what values they have. > > > > > > > > simon > > > > > > > > On Thursday, April 24, 2014 5:53:42 PM UTC+2, Luca wrote: > > > > > > > > > Of course the map is wrong. here's the right one > > > > > > > > > > ES map: > > > > > { > > > > > "mapName": { > > > > > "\_all" : {"enabled" : false}, > > > > > "properties": { > > > > > "uselessField1": { > > > > > "type": "boolean", > > > > > "index": "no" > > > > > }, > > > > > "uselessField2": { > > > > > "type": "long", > > > > > "index": "no" > > > > > }, > > > > > ... > > > > > "toBeSearched1": { > > > > > "type": "string", > > > > > "index": "de\_analyzer" > > > > > }, > > > > > "toBeSearched2": { > > > > > "type": "string", > > > > > "index": "de\_analyzer" > > > > > }, > > > > > "importantId": { > > > > > "type": "string", > > > > > "index": "not\_analyzed" > > > > > }, > > > > > "importantField": { > > > > > "type": "string", > > > > > "index": "not\_analyzed" > > > > > } > > > > > } > > > > > } > > > > > } > > > > > > > > > > -- > > > > > You received this message because you are subscribed to a topic in the > > > > > Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit > > > > > [https://groups.google.com/d/topic/elasticsearch/\_7lP\_AJrk-c/unsubscribe](https://groups.google.com/d/topic/elasticsearch/_7lP_AJrk-c/unsubscribe) > > > > > . > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4d710143-89d4-4fc8-9b2e-0d6bf28bf2a7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4d710143-89d4-4fc8-9b2e-0d6bf28bf2a7%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Luca\_Puccini](https://avatars.discourse-cdn.com/v4/letter/l/9dc877/32.png) [@Luca\_Puccini](https://discuss.elastic.co/u/Luca_Puccini)\ **Post date:** [April 28, 2014, 1:00pm UTC](https://discuss.elastic.co/t/inexplicable-behavior-using-terms-filter/17179/8 "2014-04-28T13:00:20Z")
Hello Luca and Simon, thank you very much for your answer, after a few tests everything seems to work fine now. I tried that solution by myself but I must have made some mistakes back then. By the way, is there a reason why the query works if you just filter using "term" and does not work if you use "terms"? Shouldn't simply don't work in both cases, due to the fact that the mapping is wrong? Luca -- Luca Puccini dev @ [mixd.tv](http://mixd.tv) [www.mixd.tv](http://www.mixd.tv) [http://www.crowdengineering.com](http://www.crowdengineering.com) mailto: [luca.puccini@mixd.tv](mailto:luca.puccini@mixd.tv) [luca.puccini@crowdengineering.com](mailto:luca.puccini@crowdengineering.com) Skype: lucap283 On Fri, Apr 25, 2014 at 8:52 PM, simonw [simon.willnauer@gmail.com](mailto:simon.willnauer@gmail.com) wrote: > as my colleague Luca Cavanna said there is something wrong with your > mapping... try this: [mixed\_tv\_mapping · GitHub](https://gist.github.com/s1monw/11299421) > > this should work with your queries... > > On Friday, April 25, 2014 6:33:15 PM UTC+2, Luca Cavanna wrote: > > > Hi Luca, > > I think you should have a look at the analyze and check how your > > different fields end up in the index. If the second query doesn't return > > any result it means that the filters you added there don't match any > > document. > > > > What I also noticed is that in your mapping you refer to importantId and > > so on, but they are under the importantInfo object in your documents. Have > > a look at your generated mapping too using the get mapping api and see if > > what you get back is what you submitted or maybe it contains additional > > fields with default text analysis. > > > > Cheers > > Luca > > > > On Friday, April 25, 2014 5:39:22 PM UTC+2, Luca wrote: > > > > > Attached you can find a script that basically create an index, a map and > > > add 3 elements to that. > > > > > > the following queries returns respectively 1 (correctly) and 0 results > > > > > > { > > > "query": { > > > "filtered": { > > > "query": { > > > "multi\_match": { > > > "fields": [ > > > "importantInfo.toBeSearched1^2", > > > "importantInfo.toBeSearched2" > > > ], > > > "query": "termA" > > > } > > > }, > > > "filter": { > > > "and": [ > > > { > > > "term": { > > > "importantInfo.fieldNotInMap": true > > > } > > > } > > > ] > > > } > > > } > > > } > > > } > > > { > > > "query": { > > > "filtered": { > > > "query": { > > > "multi\_match": { > > > "fields": [ > > > "importantInfo.toBeSearched1^2", > > > "importantInfo.toBeSearched2" > > > ], > > > "query": "termA" > > > } > > > }, > > > "filter": { > > > "and": [ > > > { > > > "term": { > > > "importantInfo.fieldNotInMap": true > > > } > > > }, > > > { > > > "term" : { > > > "importantInfo.importantField": "typeA" > > > } > > > }, > > > { > > > "terms" : { > > > "importantInfo.importantId": [ "stringId2", > > > "stringId1" ] > > > } > > > } > > > ] > > > } > > > } > > > } > > > } > > > > > > As I told you before this is not 100% my problem. On my system I can > > > actually filter using _terms_ on another field (_importantField_ in the > > > above example) that is exactly of the same type of _importantId_. I > > > haven't been able to reproduce that. BTW if I could understand how to > > > manage terms in this case in order to have the expected result it would be > > > great. > > > > > > thanks > > > Luca > > > > > > On Friday, April 25, 2014 4:37:00 PM UTC+2, Luca wrote: > > > > > > > Hi Simon, thank you for your answer: > > > > > > > > the values of importantId field are a finite set of strings. Those > > > > values are 100% identical to the ones required in the query. They are not > > > > analyzed because, correct me if I'm wrong, here[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html)says that the filter works on _not > > > > analyzed_ fields. > > > > > > > > Just to be clear, suppose that importantId can assume these values: > > > > ['apple', 'pear', 'orange', 'apricot', 'banana'] > > > > I want the filter to be able to eliminate all the elements that are not > > > > ['apple', 'banana']. > > > > > > > > About the results I'm building an example, and soon as it will be > > > > running I'll send it here, hopefully by the end of the day. > > > > > > > > thanks again! > > > > Luca > > > > > > > > On Thu, Apr 24, 2014 at 6:42 PM, simonw [simon.w...@elasticsearch.com](mailto:simon.w...@elasticsearch.com)wrote: > > > > > > > > > luca, what do the values in 'importantId` look like? Are they > > > > > identical to the values you pass in? The field is not analyzed though. It > > > > > would also be helpful if you could show the results that you would expect > > > > > so we can see what values they have. > > > > > > > > > > simon > > > > > > > > > > On Thursday, April 24, 2014 5:53:42 PM UTC+2, Luca wrote: > > > > > > > > > > > Of course the map is wrong. here's the right one > > > > > > > > > > > > ES map: > > > > > > { > > > > > > "mapName": { > > > > > > "\_all" : {"enabled" : false}, > > > > > > "properties": { > > > > > > "uselessField1": { > > > > > > "type": "boolean", > > > > > > "index": "no" > > > > > > }, > > > > > > "uselessField2": { > > > > > > "type": "long", > > > > > > "index": "no" > > > > > > }, > > > > > > ... > > > > > > "toBeSearched1": { > > > > > > "type": "string", > > > > > > "index": "de\_analyzer" > > > > > > }, > > > > > > "toBeSearched2": { > > > > > > "type": "string", > > > > > > "index": "de\_analyzer" > > > > > > }, > > > > > > "importantId": { > > > > > > "type": "string", > > > > > > "index": "not\_analyzed" > > > > > > }, > > > > > > "importantField": { > > > > > > "type": "string", > > > > > > "index": "not\_analyzed" > > > > > > } > > > > > > } > > > > > > } > > > > > > } > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to a topic in the > > > > > > Google Groups "elasticsearch" group. > > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > > topic/elasticsearch/\_7lP\_AJrk-c/unsubscribe. > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > > msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c% > > > > > > [40googlegroups.com](http://40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e77ccaee-de1e-4e47-8729-38a26eed1a4c%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/\_7lP\_AJrk-c/unsubscribe](https://groups.google.com/d/topic/elasticsearch/_7lP_AJrk-c/unsubscribe). > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/4d710143-89d4-4fc8-9b2e-0d6bf28bf2a7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4d710143-89d4-4fc8-9b2e-0d6bf28bf2a7%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/4d710143-89d4-4fc8-9b2e-0d6bf28bf2a7%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/4d710143-89d4-4fc8-9b2e-0d6bf28bf2a7%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CA%2B0cMO%2BKjdtW9V6yds4pDurRn6mscZZfXM7Tq-4iqQPVKOqV7g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CA%2B0cMO%2BKjdtW9V6yds4pDurRn6mscZZfXM7Tq-4iqQPVKOqV7g%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:33am UTC](https://discuss.elastic.co/t/inexplicable-behavior-using-terms-filter/17179/9 "2017-07-06T01:33:06Z")