# Records neglected in search result **URL:** \ **Category:** Elasticsearch\ **Created:** [September 15, 2012, 8:19am UTC](https://discuss.elastic.co/t/records-neglected-in-search-result/9034 "2012-09-15T08:19:21Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Kartavya](https://avatars.discourse-cdn.com/v4/letter/k/65b543/32.png) [@Kartavya](https://discuss.elastic.co/u/Kartavya)\ **Post date:** [September 15, 2012, 8:19am UTC](https://discuss.elastic.co/t/records-neglected-in-search-result/9034/1 "2012-09-15T08:19:21Z")
Hi I am getting few record missing in search result even if the record like a hit for query. For Example: Record in index is like- {"msg":"abc def"} When I am searching for "abc" the record in not included in result but at the same time when I am searching for "abc AND def" then record appear in the result. I am wondering whether there is some criteria in ElasticSearch based on score or something else then ignore some records in result. Please help. Thanks, Pulkit Agrawal -- ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [September 15, 2012, 6:43pm UTC](https://discuss.elastic.co/t/records-neglected-in-search-result/9034/2 "2012-09-15T18:43:38Z")
Hi, I think you can get better help if you provide culr recreation. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Regards, Lukas On Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal [pulkitdotcom@gmail.com](mailto:pulkitdotcom@gmail.com)wrote: > Hi > > I am getting few record missing in search result even if the record like a > hit for query. > > For Example: > Record in index is like- {"msg":"abc def"} > When I am searching for "abc" the record in not included in result but at > the same time when I am searching for "abc AND def" then record appear in > the result. > > I am wondering whether there is some criteria in Elasticsearch based on > score or something else then ignore some records in result. > > Please help. > > Thanks, > Pulkit Agrawal > > -- -- ---
**Author:** ![Kartavya](https://avatars.discourse-cdn.com/v4/letter/k/65b543/32.png) [@Kartavya](https://discuss.elastic.co/u/Kartavya)\ **Post date:** [September 17, 2012, 10:11am UTC](https://discuss.elastic.co/t/records-neglected-in-search-result/9034/3 "2012-09-17T10:11:08Z")
Sure Lukas _My first query:-_ curl -XGET 'localhost:9200XXX/YYYt/\_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage))"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum\_number\_should\_match":6} } , "filter": {"range":{"CreatedAt": {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":[" bit.ly","rt","http","[t.co](http://t.co)","i","you","my","your""]}}}}' \* \* _It return count for term "great" as 207 but When I run the same query with term "great" in AND the count changes to 322 but it should be 207. Second query is a below._ curl -XGET 'localhost:9200XXX/YYYt/\_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage)) _AND (great)_"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum\_number\_should\_match":6} } , "filter": {"range":{"CreatedAt": {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":[" bit.ly","rt","http","[t.co](http://t.co)","i","you","my","your""]}}}}' _Now term "great" count changes to 302._ \* \* _Please suggest something._ \* \* _Thanks,_ \* \* On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote: > Hi, > > I think you can get better help if you provide culr recreation. > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) > > Regards, > Lukas > > On Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal [pulkitdotcom@gmail.com](mailto:pulkitdotcom@gmail.com)wrote: > > > Hi > > > > I am getting few record missing in search result even if the record like > > a hit for query. > > > > For Example: > > Record in index is like- {"msg":"abc def"} > > When I am searching for "abc" the record in not included in result but at > > the same time when I am searching for "abc AND def" then record appear in > > the result. > > > > I am wondering whether there is some criteria in Elasticsearch based on > > score or something else then ignore some records in result. > > > > Please help. > > > > Thanks, > > Pulkit Agrawal > > > > -- > > -- -- ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [September 17, 2012, 10:19am UTC](https://discuss.elastic.co/t/records-neglected-in-search-result/9034/4 "2012-09-17T10:19:09Z")
Hi, Here is an example of a full curl recreation. [https://gist.github.com/2965777](https://gist.github.com/2965777) Please provide something similar. It will be easier to replay your case. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 17 sept. 2012 à 12:11, Pulkit Agrawal [pulkitdotcom@gmail.com](mailto:pulkitdotcom@gmail.com) a écrit : Sure Lukas My first query:- curl -XGET 'localhost:9200XXX/YYYt/\_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage))"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum\_number\_should\_match":6} } , "filter": {"range":{"CreatedAt": {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["bit.ly","rt","http","t.co","i","you","my","your""]}}}}' It return count for term "great" as 207 but When I run the same query with term "great" in AND the count changes to 322 but it should be 207. Second query is a below. curl -XGET 'localhost:9200XXX/YYYt/\_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage)) AND (great)"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum\_number\_should\_match":6} } , "filter": {"range":{"CreatedAt": {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["bit.ly","rt","http","t.co","i","you","my","your""]}}}}' Now term "great" count changes to 302. Please suggest something. Thanks, On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote: Hi, I think you can get better help if you provide culr recreation. [http://www.elasticsearch.org/help/](http://www.elasticsearch.org/help/) Regards, Lukas On Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal [pulkitdotcom@gmail.com](mailto:pulkitdotcom@gmail.com) wrote: Hi I am getting few record missing in search result even if the record like a hit for query. For Example: Record in index is like- {"msg":"abc def"} When I am searching for "abc" the record in not included in result but at the same time when I am searching for "abc AND def" then record appear in the result. I am wondering whether there is some criteria in ElasticSearch based on score or something else then ignore some records in result. Please help. ## Thanks, Pulkit Agrawal -- -- -- ---
**Author:** ![Kartavya](https://avatars.discourse-cdn.com/v4/letter/k/65b543/32.png) [@Kartavya](https://discuss.elastic.co/u/Kartavya)\ **Post date:** [September 17, 2012, 10:30am UTC](https://discuss.elastic.co/t/records-neglected-in-search-result/9034/5 "2012-09-17T10:30:16Z")
Hope, it is more readable now _My first query:-_ { "query": { "filtered": { "query": { "bool": { "should": [ { "field": { "MsgData": "((gayANDMarriage) OR (MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage))" } }, { "field": { "UserGender": "FORMORU" } }, { "field": { "DataSourceTypeID": "BORTORNORDORFORVORSORRORX" } }, { "field": { "DataSourceIndustry": "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" } }, { "field": { "DataSourceBusinessArea": "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU" } }, { "field": { "polarity": "2OR1" } } ], "minimum\_number\_should\_match": 6 } }, "filter": { "range": { "CreatedAt": { "from": "2012-9-10T00: 00: 00", "to": "2012-9-15T23: 59: 59" } } } } }, "facets": { "MsgData": { "terms": { "field": "MsgData", "size": 100, "exclude": [ "bit.ly", "rt", "http", "[t.co](http://t.co)", "i", "you", "my", "your" ] } } } } \* \* _It return count for term "great" as 207 but When I run the same query with term "great" in AND the count changes to 322 but it should be 207. Second query is a below._ { "query": { "filtered": { "query": { "bool": { "should": [ { "field": { "MsgData": " ((gayANDMarriage) OR (MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage)) _AND (great)_" } }, { "field": { "UserGender": "FORMORU" } }, { "field": { "DataSourceTypeID": "BORTORNORDORFORVORSORRORX" } }, { "field": { "DataSourceIndustry": "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" } }, { "field": { "DataSourceBusinessArea": "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU" } }, { "field": { "polarity": "2OR1" } } ], "minimum\_number\_should\_match": 6 } }, "filter": { "range": { "CreatedAt": { "from": "2012-9-10T00: 00: 00", "to": "2012-9-15T23: 59: 59" } } } } }, "facets": { "MsgData": { "terms": { "field": "MsgData", "size": 100, "exclude": [ "bit.ly", "rt", "http", "[t.co](http://t.co)", "i", "you", "my", "your" ] } } } } _Now term "great" count changes to 302._ \* \* _Please suggest something._ \* \* _Thanks,_ \* \* On Mon, Sep 17, 2012 at 3:49 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Hi, > > Here is an example of a full curl recreation. > [Test case for multifield fields · GitHub](https://gist.github.com/2965777) > Please provide something similar. It will be easier to replay your case. > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 17 sept. 2012 à 12:11, Pulkit Agrawal [pulkitdotcom@gmail.com](mailto:pulkitdotcom@gmail.com) a > écrit : > > Sure Lukas > > _My first query:-_ > curl -XGET 'localhost:9200XXX/YYYt/\_search?size=0' -d > '{"query":{"filtered": > {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR > (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR > (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR > (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR > (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR > (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly > AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one > AND man AND one AND woman) OR (same AND gender AND > marriage))"}},{"field":{"UserGender":"F OR M OR > U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR > X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND > OR ITC OR MAT OR TEL OR UD OR UTL OR All OR > U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR > PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR > U"}},{"field":{"polarity":"2 OR 1"}}],"minimum\_number\_should\_match":6} } , > "filter": {"range":{"CreatedAt": > {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } > },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":[" > bit.ly","rt","http","[t.co](http://t.co)","i","you","my","your""]}}}}' > \* > \* > _It return count for term "great" as 207 but When I run the same query > with term "great" in AND the count changes to 322 but it should be 207. > Second query is a below._ > > curl -XGET 'localhost:9200XXX/YYYt/\_search?size=0' -d > '{"query":{"filtered": > {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR > (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR > (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR > (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR > (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR > (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly > AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one > AND man AND one AND woman) OR (same AND gender AND marriage)) \*AND (great) > \*"}},{"field":{"UserGender":"F OR M OR > U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR > X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND > OR ITC OR MAT OR TEL OR UD OR UTL OR All OR > U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR > PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR > U"}},{"field":{"polarity":"2 OR 1"}}],"minimum\_number\_should\_match":6} } , > "filter": {"range":{"CreatedAt": > {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } > },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":[" > bit.ly","rt","http","[t.co](http://t.co)","i","you","my","your""]}}}}' > > _Now term "great" count changes to 302._ > \* > \* > _Please suggest something._ > \* > \* > _Thanks,_ > \* > \* > > On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com)wrote: > > > Hi, > > > > I think you can get better help if you provide culr recreation. > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) > > > > Regards, > > Lukas > > > > On Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal [pulkitdotcom@gmail.com](mailto:pulkitdotcom@gmail.com)wrote: > > > > > Hi > > > > > > I am getting few record missing in search result even if the record like > > > a hit for query. > > > > > > For Example: > > > Record in index is like- {"msg":"abc def"} > > > When I am searching for "abc" the record in not included in result but > > > at the same time when I am searching for "abc AND def" then record appear > > > in the result. > > > > > > I am wondering whether there is some criteria in Elasticsearch based on > > > score or something else then ignore some records in result. > > > > > > Please help. > > > > > > Thanks, > > > Pulkit Agrawal > > > > > > -- > > > > -- > > -- > > -- -- ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [September 17, 2012, 10:46am UTC](https://discuss.elastic.co/t/records-neglected-in-search-result/9034/6 "2012-09-17T10:46:44Z")
I think you did not understand what Lukas and I are saying ! If I run your query on my ES instance. It won't work. How can we help you if we are able to reproduce your use case? We don't know your settings, your mappings, your docs... -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 17 sept. 2012 à 12:30, Pulkit Agrawal [pulkitdotcom@gmail.com](mailto:pulkitdotcom@gmail.com) a écrit : Hope, it is more readable now My first query:- { "query": { "filtered": { "query": { "bool": { "should": [ { "field": { "MsgData": "((gayANDMarriage) OR (MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage))" } }, { "field": { "UserGender": "FORMORU" } }, { "field": { "DataSourceTypeID": "BORTORNORDORFORVORSORRORX" } }, { "field": { "DataSourceIndustry": "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" } }, { "field": { "DataSourceBusinessArea": "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU" } }, { "field": { "polarity": "2OR1" } } ], "minimum\_number\_should\_match": 6 } }, "filter": { "range": { "CreatedAt": { "from": "2012-9-10T00: 00: 00", "to": "2012-9-15T23: 59: 59" } } } } }, "facets": { "MsgData": { "terms": { "field": "MsgData", "size": 100, "exclude": [ "bit.ly", "rt", "http", "t.co", "i", "you", "my", "your" ] } } } } It return count for term "great" as 207 but When I run the same query with term "great" in AND the count changes to 322 but it should be 207. Second query is a below. { "query": { "filtered": { "query": { "bool": { "should": [ { "field": { "MsgData": " ((gayANDMarriage) OR (MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage)) AND (great)" } }, { "field": { "UserGender": "FORMORU" } }, { "field": { "DataSourceTypeID": "BORTORNORDORFORVORSORRORX" } }, { "field": { "DataSourceIndustry": "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" } }, { "field": { "DataSourceBusinessArea": "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU" } }, { "field": { "polarity": "2OR1" } } ], "minimum\_number\_should\_match": 6 } }, "filter": { "range": { "CreatedAt": { "from": "2012-9-10T00: 00: 00", "to": "2012-9-15T23: 59: 59" } } } } }, "facets": { "MsgData": { "terms": { "field": "MsgData", "size": 100, "exclude": [ "bit.ly", "rt", "http", "t.co", "i", "you", "my", "your" ] } } } } Now term "great" count changes to 302. Please suggest something. Thanks, On Mon, Sep 17, 2012 at 3:49 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: Hi, Here is an example of a full curl recreation. [https://gist.github.com/2965777](https://gist.github.com/2965777) Please provide something similar. It will be easier to replay your case. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 17 sept. 2012 à 12:11, Pulkit Agrawal [pulkitdotcom@gmail.com](mailto:pulkitdotcom@gmail.com) a écrit : Sure Lukas My first query:- curl -XGET 'localhost:9200XXX/YYYt/\_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage))"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum\_number\_should\_match":6} } , "filter": {"range":{"CreatedAt": {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["bit.ly","rt","http","t.co","i","you","my","your""]}}}}' It return count for term "great" as 207 but When I run the same query with term "great" in AND the count changes to 322 but it should be 207. Second query is a below. curl -XGET 'localhost:9200XXX/YYYt/\_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage)) AND (great)"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum\_number\_should\_match":6} } , "filter": {"range":{"CreatedAt": {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["bit.ly","rt","http","t.co","i","you","my","your""]}}}}' Now term "great" count changes to 302. Please suggest something. Thanks, On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote: Hi, I think you can get better help if you provide culr recreation. [http://www.elasticsearch.org/help/](http://www.elasticsearch.org/help/) Regards, Lukas On Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal [pulkitdotcom@gmail.com](mailto:pulkitdotcom@gmail.com) wrote: Hi I am getting few record missing in search result even if the record like a hit for query. For Example: Record in index is like- {"msg":"abc def"} When I am searching for "abc" the record in not included in result but at the same time when I am searching for "abc AND def" then record appear in the result. I am wondering whether there is some criteria in ElasticSearch based on score or something else then ignore some records in result. Please help. ## Thanks, Pulkit Agrawal -- -- -- -- -- ---
**Author:** ![Kartavya](https://avatars.discourse-cdn.com/v4/letter/k/65b543/32.png) [@Kartavya](https://discuss.elastic.co/u/Kartavya)\ **Post date:** [September 17, 2012, 10:56am UTC](https://discuss.elastic.co/t/records-neglected-in-search-result/9034/7 "2012-09-17T10:56:03Z")
here is my mapping of index curl -XPUT 'localhost:9200/inn/innt/\_mapping' -d '{ "innt": { "properties": { "FBLikeCount": { "type": "integer" }, "FBShareCount": { "type": "integer" }, "FBRecommendCount": { "type": "integer" }, "FBTotalCount": { "type": "integer" }, "FBClickCount": { "type": "integer" }, "TWShareCount": { "type": "integer" }, "NoOfComments": { "type": "integer" }, "VideoWatchCount": { "type": "integer" }, "AuthorNumberOfResult1": { "type": "integer" }, "polarity": { "type": "multi\_field", "fields": { "polarity": { "type": "string" }, "polarityint": { "type": "integer" } } } } } }}' Here is my sample document: { "\_id": "245745919428014081", "\_rev": "1-e863bfb49a313b61f1a07cf3224dad49", "SystemCreatedAt": "2012-09-12T05:13:32", "MsgData": "I shoulda taken my medicine. Caught a bad case of feelings.", "IsTweet": "Y", "CreatedAt": "2012-9-12T04:49:27", "Retweet": "N", "SystemModifiedAt": "2012-09-12T05:13:32", "ReplyTweetID": "", "ReplyUserID": "", "IsEnglish": "Y", "polarity": "2", "MessageEfluencerIndex": 91409.825, "BusinessAreaConfiguration": "UD", "Feature": "", "LifeStyle": " HLT", "Source": "[Echofon](http://www.echofon.com/)", "AuthorLocationKnow": "Y", "AuthorLocationContinentCode": "NORTHAMERICA", "AuthorLocationCountryCode": "US", "AuthorLocationStateCode": " WY", "GeoEnabled": "N", "PlaceFullName": "", "PlaceName": "", "TweetContry": "", "TweetCountryCode": "", "TweetLatitude": "", "TweetLongigttude": "", "UserID": "23914462", "UserName": "Proto\_Zsay", "ToUserID": "", "ToUserName": "", "IsPopular": "N", "UserGender": "M", "Name": "Pres. Chill Clinton", "Location": "Byron Bay, NSW", "UserEfluncerIndex": 18281.765, "AuthorLocationRegionCode": "W", "RetweetCount": "0", "DataSourceTypeID": "T", "DataSourceIndustry": "All", "DataSourceBusinessArea": "All", "DataSourceCoverageArea": "All" } Other than that there is no change in setting. I am using default Elasticsearch setting of 0.18.7. Let me know, if you are looking for any other information. I am happy to provide the same. Thanks, On Mon, Sep 17, 2012 at 4:16 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > I think you did not understand what Lukas and I are saying ! > > If I run your query on my ES instance. It won't work. > How can we help you if we are able to reproduce your use case? > > We don't know your settings, your mappings, your docs... > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 17 sept. 2012 à 12:30, Pulkit Agrawal [pulkitdotcom@gmail.com](mailto:pulkitdotcom@gmail.com) a > écrit : > > Hope, it is more readable now > > _My first query:-_ > { > "query": { > "filtered": { > "query": { > "bool": { > "should": [ > { > "field": { > "MsgData": "((gayANDMarriage) OR > (MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR > (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR > (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR > (Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR > (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly > AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one > AND man AND one AND woman) OR (same AND gender AND marriage))" > } > }, > { > "field": { > "UserGender": "FORMORU" > } > }, > { > "field": { > "DataSourceTypeID": > "BORTORNORDORFORVORSORRORX" > } > }, > { > "field": { > "DataSourceIndustry": > "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" > } > }, > { > "field": { > "DataSourceBusinessArea": > "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU" > } > }, > { > "field": { > "polarity": "2OR1" > } > } > ], > "minimum\_number\_should\_match": 6 > } > }, > "filter": { > "range": { > "CreatedAt": { > "from": "2012-9-10T00: 00: 00", > "to": "2012-9-15T23: 59: 59" > } > } > } > } > }, > "facets": { > "MsgData": { > "terms": { > "field": "MsgData", > "size": 100, > "exclude": [ > "bit.ly", > "rt", > "http", > "[t.co](http://t.co)", > "i", > "you", > "my", > "your" > ] > } > } > } > } > \* > \* > _It return count for term "great" as 207 but When I run the same query > with term "great" in AND the count changes to 322 but it should be 207. > Second query is a below._ > > { > "query": { > "filtered": { > "query": { > "bool": { > "should": [ > { > "field": { > "MsgData": " ((gayANDMarriage) OR > (MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR > (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR > (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR > (Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR > (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly > AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one > AND man AND one AND woman) OR (same AND gender AND marriage)) \*AND (great) > \*" > } > }, > { > "field": { > "UserGender": "FORMORU" > } > }, > { > "field": { > "DataSourceTypeID": > "BORTORNORDORFORVORSORRORX" > } > }, > { > "field": { > "DataSourceIndustry": > "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" > } > }, > { > "field": { > "DataSourceBusinessArea": > "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU" > } > }, > { > "field": { > "polarity": "2OR1" > } > } > ], > "minimum\_number\_should\_match": 6 > } > }, > "filter": { > "range": { > "CreatedAt": { > "from": "2012-9-10T00: 00: 00", > "to": "2012-9-15T23: 59: 59" > } > } > } > } > }, > "facets": { > "MsgData": { > "terms": { > "field": "MsgData", > "size": 100, > "exclude": [ > "bit.ly", > "rt", > "http", > "[t.co](http://t.co)", > "i", > "you", > "my", > "your" > ] > } > } > } > } > > _Now term "great" count changes to 302._ > \* > \* > _Please suggest something._ > \* > \* > _Thanks,_ > \* > \* > > On Mon, Sep 17, 2012 at 3:49 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > > > Hi, > > > > Here is an example of a full curl recreation. > > [Test case for multifield fields · GitHub](https://gist.github.com/2965777) > > Please provide something similar. It will be easier to replay your case. > > > > -- > > David 😉 > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > Le 17 sept. 2012 à 12:11, Pulkit Agrawal [pulkitdotcom@gmail.com](mailto:pulkitdotcom@gmail.com) a > > écrit : > > > > Sure Lukas > > > > _My first query:-_ > > curl -XGET 'localhost:9200XXX/YYYt/\_search?size=0' -d > > '{"query":{"filtered": > > {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR > > (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR > > (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR > > (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR > > (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR > > (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly > > AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one > > AND man AND one AND woman) OR (same AND gender AND > > marriage))"}},{"field":{"UserGender":"F OR M OR > > U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR > > X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND > > OR ITC OR MAT OR TEL OR UD OR UTL OR All OR > > U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR > > PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR > > U"}},{"field":{"polarity":"2 OR 1"}}],"minimum\_number\_should\_match":6} } , > > "filter": {"range":{"CreatedAt": > > {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } > > },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":[" > > bit.ly","rt","http","[t.co](http://t.co)","i","you","my","your""]}}}}' > > \* > > \* > > _It return count for term "great" as 207 but When I run the same query > > with term "great" in AND the count changes to 322 but it should be 207. > > Second query is a below._ > > > > curl -XGET 'localhost:9200XXX/YYYt/\_search?size=0' -d > > '{"query":{"filtered": > > {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR > > (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR > > (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR > > (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR > > (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR > > (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly > > AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one > > AND man AND one AND woman) OR (same AND gender AND marriage)) _AND > > (great)_"}},{"field":{"UserGender":"F OR M OR > > U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR > > X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND > > OR ITC OR MAT OR TEL OR UD OR UTL OR All OR > > U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR > > PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR > > U"}},{"field":{"polarity":"2 OR 1"}}],"minimum\_number\_should\_match":6} } , > > "filter": {"range":{"CreatedAt": > > {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } > > },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":[" > > bit.ly","rt","http","[t.co](http://t.co)","i","you","my","your""]}}}}' > > > > _Now term "great" count changes to 302._ > > \* > > \* > > _Please suggest something._ > > \* > > \* > > _Thanks,_ > > \* > > \* > > > > On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com)wrote: > > > > > Hi, > > > > > > I think you can get better help if you provide culr recreation. > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) > > > > > > Regards, > > > Lukas > > > > > > On Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal \<[pulkitdotcom@gmail.com](mailto:pulkitdotcom@gmail.com) > > > > > > > wrote: > > > > > > > Hi > > > > > > > > I am getting few record missing in search result even if the record > > > > like a hit for query. > > > > > > > > For Example: > > > > Record in index is like- {"msg":"abc def"} > > > > When I am searching for "abc" the record in not included in result but > > > > at the same time when I am searching for "abc AND def" then record appear > > > > in the result. > > > > > > > > I am wondering whether there is some criteria in Elasticsearch based on > > > > score or something else then ignore some records in result. > > > > > > > > Please help. > > > > > > > > Thanks, > > > > Pulkit Agrawal > > > > > > > > -- > > > > > > -- > > > > -- > > > > -- > > -- > > -- -- ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [September 17, 2012, 11:03am UTC](https://discuss.elastic.co/t/records-neglected-in-search-result/9034/8 "2012-09-17T11:03:30Z")
Hi Pulkit Have a look at [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help) It explains how to provide code samples to make it easy for others to debug problems. If we can't copy and paste something, then we are unlikely to try it. > Other than that there is no change in setting. I am using default > Elasticsearch setting of 0.18.7. Also, I would upgrade elasticsearch. A number of bugs have been fixed since 0.18.7 clint > Let me know, if you are looking for any other information. I am happy > to provide the same. > > Thanks, > > On Mon, Sep 17, 2012 at 4:16 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > I think you did not understand what Lukas and I are saying ! > > ``` > If I run your query on my ES instance. It won't work. > How can we help you if we are able to reproduce your use case? > > > We don't know your settings, your mappings, your docs... > > -- > David ;-) > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > Le 17 sept. 2012 à 12:30, Pulkit Agrawal > a écrit : > > > > Hope, it is more readable now > > > My first query:- > { > "query": { > "filtered": { > "query": { > "bool": { > "should": [ > { > "field": { > "MsgData": "((gayANDMarriage) > OR (MarriageANDequality) OR (LGBTANDmarriage) OR > (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR > (Marriageequality AND New AND Jersey) OR (Marriageequality AND > Minnesota) OR (Marriageequality AND New AND YORk) OR > (Marriageequality AND MarylAND ) OR (Marriage AND quality AND > Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND > DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR > (Marriageequality AND ban) OR (one AND man AND one AND woman) > OR (same AND gender AND marriage))" > } > }, > { > "field": { > "UserGender": "FORMORU" > } > }, > { > "field": { > "DataSourceTypeID": > "BORTORNORDORFORVORSORRORX" > } > }, > { > "field": { > "DataSourceIndustry": > "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" > } > }, > { > "field": { > "DataSourceBusinessArea": > "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU" > } > }, > { > "field": { > "polarity": "2OR1" > } > } > ], > "minimum_number_should_match": 6 > } > }, > "filter": { > "range": { > "CreatedAt": { > "from": "2012-9-10T00: 00: 00", > "to": "2012-9-15T23: 59: 59" > } > } > } > } > }, > "facets": { > "MsgData": { > "terms": { > "field": "MsgData", > "size": 100, > "exclude": [ > "bit.ly", > "rt", > "http", > "t.co", > "i", > "you", > "my", > "your" > ] > } > } > } > } > > > It return count for term "great" as 207 but When I run the > same query with term "great" in AND the count changes to 322 > but it should be 207. Second query is a below. > > > { > "query": { > "filtered": { > "query": { > "bool": { > "should": [ > { > "field": { > "MsgData": " ((gayANDMarriage) > OR (MarriageANDequality) OR (LGBTANDmarriage) OR > (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR > (Marriageequality AND New AND Jersey) OR (Marriageequality AND > Minnesota) OR (Marriageequality AND New AND YORk) OR > (Marriageequality AND MarylAND ) OR (Marriage AND quality AND > Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND > DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR > (Marriageequality AND ban) OR (one AND man AND one AND woman) > OR (same AND gender AND marriage)) AND (great)" > } > }, > { > "field": { > "UserGender": "FORMORU" > } > }, > { > "field": { > "DataSourceTypeID": > "BORTORNORDORFORVORSORRORX" > } > }, > { > "field": { > "DataSourceIndustry": > "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" > } > }, > { > "field": { > "DataSourceBusinessArea": > "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU" > } > }, > { > "field": { > "polarity": "2OR1" > } > } > ], > "minimum_number_should_match": 6 > } > }, > "filter": { > "range": { > "CreatedAt": { > "from": "2012-9-10T00: 00: 00", > "to": "2012-9-15T23: 59: 59" > } > } > } > } > }, > "facets": { > "MsgData": { > "terms": { > "field": "MsgData", > "size": 100, > "exclude": [ > "bit.ly", > "rt", > "http", > "t.co", > "i", > "you", > "my", > "your" > ] > } > } > } > } > > > Now term "great" count changes to 302. > > > Please suggest something. > > > Thanks, > > > > On Mon, Sep 17, 2012 at 3:49 PM, David Pilato > wrote: > Hi, > > > Here is an example of a full curl > recreation. https://gist.github.com/2965777 > Please provide something similar. It will be easier to > replay your case. > > -- > David ;-) > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > Le 17 sept. 2012 à 12:11, Pulkit Agrawal > a écrit : > > > > > > Sure Lukas > > > My first query:- > curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d > '{"query":{"filtered": > {"query":{"bool":{"should":[{"field":{"MsgData":"((gay > AND Marriage) OR (Marriage AND equality) OR (LGBT AND > marriage) OR (Samesex AND marriage) OR (Prop8) OR > (Samesex) OR (Marriageequality AND New AND Jersey) OR > (Marriageequality AND Minnesota) OR (Marriageequality > AND New AND YORk) OR (Marriageequality AND MarylAND) > OR (Marriage AND Equality AND Act) OR (Connie AND > Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) > OR (Molly AND McKay) OR (Davina AND Kotulski) OR > (Marriageequality AND ban) OR (one AND man AND one AND > woman) OR (same AND gender AND > marriage))"}},{"field":{"UserGender":"F OR M OR > U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR > F OR V OR S OR R OR > X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY > OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL > OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS > OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI > OR MA OR OPE OR PLT OR TNT OR UD OR All OR > U"}},{"field":{"polarity":"2 OR > 1"}}],"minimum_number_should_match":6} } , "filter": > {"range":{"CreatedAt": > {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["bit.ly","rt","http","t.co","i","you","my","your""]}}}}' > > > It return count for term "great" as 207 but When I run > the same query with term "great" in AND the count > changes to 322 but it should be 207. Second query is a > below. > > > curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d > '{"query":{"filtered": > {"query":{"bool":{"should":[{"field":{"MsgData":"((gay > AND Marriage) OR (Marriage AND equality) OR (LGBT AND > marriage) OR (Samesex AND marriage) OR (Prop8) OR > (Samesex) OR (Marriageequality AND New AND Jersey) OR > (Marriageequality AND Minnesota) OR (Marriageequality > AND New AND YORk) OR (Marriageequality AND MarylAND) > OR (Marriage AND Equality AND Act) OR (Connie AND > Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) > OR (Molly AND McKay) OR (Davina AND Kotulski) OR > (Marriageequality AND ban) OR (one AND man AND one AND > woman) OR (same AND gender AND marriage)) AND > (great)"}},{"field":{"UserGender":"F OR M OR > U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR > F OR V OR S OR R OR > X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY > OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL > OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS > OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI > OR MA OR OPE OR PLT OR TNT OR UD OR All OR > U"}},{"field":{"polarity":"2 OR > 1"}}],"minimum_number_should_match":6} } , "filter": > {"range":{"CreatedAt": > {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["bit.ly","rt","http","t.co","i","you","my","your""]}}}}' > > > Now term "great" count changes to 302. > > > Please suggest something. > > > Thanks, > > > > On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček > wrote: > Hi, > > > I think you can get better help if you provide > culr recreation. > http://www.elasticsearch.org/help/ > > > Regards, > Lukas > > On Sat, Sep 15, 2012 at 10:19 AM, Pulkit > Agrawal wrote: > > Hi > > > I am getting few record missing in > search result even if the record like > a hit for query. > > > For Example: > Record in index is like- {"msg":"abc > def"} > When I am searching for "abc" the > record in not included in result but > at the same time when I am searching > for "abc AND def" then record appear > in the result. > > > I am wondering whether there is some > criteria in ElasticSearch based on > score or something else then ignore > some records in result. > > > Please help. > > > Thanks, > Pulkit Agrawal > -- > > > > > > -- > > > > -- > > > > -- > > > > -- > > ``` -- ---
**Author:** ![Kartavya](https://avatars.discourse-cdn.com/v4/letter/k/65b543/32.png) [@Kartavya](https://discuss.elastic.co/u/Kartavya)\ **Post date:** [September 17, 2012, 12:15pm UTC](https://discuss.elastic.co/t/records-neglected-in-search-result/9034/9 "2012-09-17T12:15:46Z")
Thanks. I found same issue here > > > I've got an issue with facet counts that I've managed to simplify into a re-crea…table example (attached). > > When searching an index I get a facet count for 'fabric' as 12 however when I then filter on that attribute it increases to 13, not sure how this is possible as by adding a must query I can surely only decrease the facet counts. > > Attached is the script which should recreate the issue in a test\_bug index, it will insert 118 documents and run a query where the "fabric" facet comes out at 12. > > Query looks like: > > \`\`\` json > { > "query": { > "bool": { > "must": \[ > { > "query\_string": { > "default\_operator": "AND", > "fields": \[ > "\_all" > \], > "query": "double" > } > } > \] > } > }, > "facets": { > "styles": { > "terms": { > "field": "product\_styles" > } > } > }, > "size": 1 > } > \`\`\` > > Facet comes out as: > > \`\`\` json > "facets" : { > "styles" : { > "\_type" : "terms", > "missing" : 11, > "total" : 243, > "other" : 141, > "terms" : \[ { > "term" : "beds", > "count" : 19 > }, { > "term" : "fabric", > "count" : 12 > }, { > "term" : "products", > "count" : 11 > }, { > "term" : "new", > "count" : 11 > }, { > "term" : "luxury", > "count" : 10 > }, { > "term" : "furniture", > "count" : 10 > }, { > "term" : "upholstered", > "count" : 8 > }, { > "term" : "kingsize", > "count" : 7 > }, { > "term" : "double", > "count" : 7 > }, { > "term" : "bedroom", > "count" : 7 > } \] > } > \`\`\` > > When adding a filter on the fabric like this: > > \`\`\` json > { > "query": { > "bool": { > "must": \[ > { > "query\_string": { > "default\_operator": "AND", > "fields": \[ > "\_all" > \], > "query": "double" > } > }, > { > "term": { > "product\_styles": "fabric" > } > } > \] > } > }, > "facets": { > "styles": { > "terms": { > "field": "product\_styles" > } > } > }, > "size": 1 > } > \`\`\` > > the fabric facet then increases to 13: > > \`\`\` json > "facets" : { > "styles" : { > "\_type" : "terms", > "missing" : 0, > "total" : 186, > "other" : 97, > "terms" : \[ { > "term" : "fabric", > "count" : 13 > }, { > "term" : "beds", > "count" : 12 > }, { > "term" : "upholstered", > "count" : 9 > }, { > "term" : "luxury", > "count" : 9 > }, { > "term" : "products", > "count" : 8 > }, { > "term" : "new", > "count" : 8 > }, { > "term" : "kingsize", > "count" : 8 > }, { > "term" : "double", > "count" : 8 > }, { > "term" : "furniture", > "count" : 7 > }, { > "term" : "bedroom", > "count" : 7 > } \] > } > \`\`\` > > Hopefully I've explained everything and you can also recreate this. I'm using elasticsearch-0.19.1 on Mac OS X Lion. > > Script for recreating data is at https://gist.github.com/2283964 Now, can anybody help me? I can not use single shard. What else I can do? On Mon, Sep 17, 2012 at 4:33 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > Hi Pulkit > > Have a look at [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help) > > It explains how to provide code samples to make it easy for others to > debug problems. If we can't copy and paste something, then we are > unlikely to try it. > > > Other than that there is no change in setting. I am using default > > Elasticsearch setting of 0.18.7. > > Also, I would upgrade elasticsearch. A number of bugs have been fixed > since 0.18.7 > > clint > > > Let me know, if you are looking for any other information. I am happy > > to provide the same. > > > > Thanks, > > > > On Mon, Sep 17, 2012 at 4:16 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > > I think you did not understand what Lukas and I are saying ! > > > > ``` > > If I run your query on my ES instance. It won't work. > > How can we help you if we are able to reproduce your use case? > > > > We don't know your settings, your mappings, your docs... > > > > -- > > David ;-) > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > Le 17 sept. 2012 à 12:30, Pulkit Agrawal > > a écrit : > > > > Hope, it is more readable now > > > > My first query:- > > { > > "query": { > > "filtered": { > > "query": { > > "bool": { > > "should": [ > > { > > "field": { > > "MsgData": "((gayANDMarriage) > > OR (MarriageANDequality) OR (LGBTANDmarriage) OR > > (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR > > (Marriageequality AND New AND Jersey) OR (Marriageequality AND > > Minnesota) OR (Marriageequality AND New AND YORk) OR > > (Marriageequality AND MarylAND ) OR (Marriage AND quality AND > > Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND > > DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR > > (Marriageequality AND ban) OR (one AND man AND one AND woman) > > OR (same AND gender AND marriage))" > > } > > }, > > { > > "field": { > > "UserGender": "FORMORU" > > } > > }, > > { > > "field": { > > "DataSourceTypeID": > > "BORTORNORDORFORVORSORRORX" > > } > > }, > > { > > "field": { > > "DataSourceIndustry": > > "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" > > } > > }, > > { > > "field": { > > "DataSourceBusinessArea": > > > > ``` > > "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU" > > > ``` > > } > > }, > > { > > "field": { > > "polarity": "2OR1" > > } > > } > > ], > > "minimum_number_should_match": 6 > > } > > }, > > "filter": { > > "range": { > > "CreatedAt": { > > "from": "2012-9-10T00: 00: 00", > > "to": "2012-9-15T23: 59: 59" > > } > > } > > } > > } > > }, > > "facets": { > > "MsgData": { > > "terms": { > > "field": "MsgData", > > "size": 100, > > "exclude": [ > > "bit.ly", > > "rt", > > "http", > > "t.co", > > "i", > > "you", > > "my", > > "your" > > ] > > } > > } > > } > > } > > > > It return count for term "great" as 207 but When I run the > > same query with term "great" in AND the count changes to 322 > > but it should be 207. Second query is a below. > > > > { > > "query": { > > "filtered": { > > "query": { > > "bool": { > > "should": [ > > { > > "field": { > > "MsgData": " ((gayANDMarriage) > > OR (MarriageANDequality) OR (LGBTANDmarriage) OR > > (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR > > (Marriageequality AND New AND Jersey) OR (Marriageequality AND > > Minnesota) OR (Marriageequality AND New AND YORk) OR > > (Marriageequality AND MarylAND ) OR (Marriage AND quality AND > > Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND > > DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR > > (Marriageequality AND ban) OR (one AND man AND one AND woman) > > OR (same AND gender AND marriage)) AND (great)" > > } > > }, > > { > > "field": { > > "UserGender": "FORMORU" > > } > > }, > > { > > "field": { > > "DataSourceTypeID": > > "BORTORNORDORFORVORSORRORX" > > } > > }, > > { > > "field": { > > "DataSourceIndustry": > > "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" > > } > > }, > > { > > "field": { > > "DataSourceBusinessArea": > > > > ``` > > "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU" > > > ``` > > } > > }, > > { > > "field": { > > "polarity": "2OR1" > > } > > } > > ], > > "minimum_number_should_match": 6 > > } > > }, > > "filter": { > > "range": { > > "CreatedAt": { > > "from": "2012-9-10T00: 00: 00", > > "to": "2012-9-15T23: 59: 59" > > } > > } > > } > > } > > }, > > "facets": { > > "MsgData": { > > "terms": { > > "field": "MsgData", > > "size": 100, > > "exclude": [ > > "bit.ly", > > "rt", > > "http", > > "t.co", > > "i", > > "you", > > "my", > > "your" > > ] > > } > > } > > } > > } > > > > Now term "great" count changes to 302. > > > > Please suggest something. > > > > Thanks, > > > > On Mon, Sep 17, 2012 at 3:49 PM, David Pilato > > wrote: > > Hi, > > > > Here is an example of a full curl > > recreation. https://gist.github.com/2965777 > > Please provide something similar. It will be easier to > > replay your case. > > > > -- > > David ;-) > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > Le 17 sept. 2012 à 12:11, Pulkit Agrawal > > a écrit : > > > > Sure Lukas > > > > My first query:- > > curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d > > '{"query":{"filtered": > > {"query":{"bool":{"should":[{"field":{"MsgData":"((gay > > AND Marriage) OR (Marriage AND equality) OR (LGBT AND > > marriage) OR (Samesex AND marriage) OR (Prop8) OR > > (Samesex) OR (Marriageequality AND New AND Jersey) OR > > (Marriageequality AND Minnesota) OR (Marriageequality > > AND New AND YORk) OR (Marriageequality AND MarylAND) > > OR (Marriage AND Equality AND Act) OR (Connie AND > > Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) > > OR (Molly AND McKay) OR (Davina AND Kotulski) OR > > (Marriageequality AND ban) OR (one AND man AND one AND > > woman) OR (same AND gender AND > > marriage))"}},{"field":{"UserGender":"F OR M OR > > U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR > > F OR V OR S OR R OR > > X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY > > OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL > > OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS > > OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI > > OR MA OR OPE OR PLT OR TNT OR UD OR All OR > > U"}},{"field":{"polarity":"2 OR > > 1"}}],"minimum_number_should_match":6} } , "filter": > > {"range":{"CreatedAt": > > > > ``` > > {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } > },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":[" > bit.ly","rt","http","[t.co](http://t.co)","i","you","my","your""]}}}}' > > > ``` > > It return count for term "great" as 207 but When I run > > the same query with term "great" in AND the count > > changes to 322 but it should be 207. Second query is a > > below. > > > > curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d > > '{"query":{"filtered": > > {"query":{"bool":{"should":[{"field":{"MsgData":"((gay > > AND Marriage) OR (Marriage AND equality) OR (LGBT AND > > marriage) OR (Samesex AND marriage) OR (Prop8) OR > > (Samesex) OR (Marriageequality AND New AND Jersey) OR > > (Marriageequality AND Minnesota) OR (Marriageequality > > AND New AND YORk) OR (Marriageequality AND MarylAND) > > OR (Marriage AND Equality AND Act) OR (Connie AND > > Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) > > OR (Molly AND McKay) OR (Davina AND Kotulski) OR > > (Marriageequality AND ban) OR (one AND man AND one AND > > woman) OR (same AND gender AND marriage)) AND > > (great)"}},{"field":{"UserGender":"F OR M OR > > U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR > > F OR V OR S OR R OR > > X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY > > OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL > > OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS > > OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI > > OR MA OR OPE OR PLT OR TNT OR UD OR All OR > > U"}},{"field":{"polarity":"2 OR > > 1"}}],"minimum_number_should_match":6} } , "filter": > > {"range":{"CreatedAt": > > > > ``` > > {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } > },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":[" > bit.ly","rt","http","[t.co](http://t.co)","i","you","my","your""]}}}}' > > > ``` > > Now term "great" count changes to 302. > > > > Please suggest something. > > > > Thanks, > > > > On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček > > wrote: > > Hi, > > > > I think you can get better help if you provide > > culr recreation. > > http://www.elasticsearch.org/help/ > > > > Regards, > > Lukas > > > > On Sat, Sep 15, 2012 at 10:19 AM, Pulkit > > Agrawal wrote: > > > > Hi > > > > I am getting few record missing in > > search result even if the record like > > a hit for query. > > > > For Example: > > Record in index is like- {"msg":"abc > > def"} > > When I am searching for "abc" the > > record in not included in result but > > at the same time when I am searching > > for "abc AND def" then record appear > > in the result. > > > > I am wondering whether there is some > > criteria in ElasticSearch based on > > score or something else then ignore > > some records in result. > > > > Please help. > > > > Thanks, > > Pulkit Agrawal > > -- > > > > -- > > > > -- > > > > -- > > > > -- > > > > ``` > > -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:12am UTC](https://discuss.elastic.co/t/records-neglected-in-search-result/9034/10 "2017-07-06T03:12:36Z")