Hi
I am getting few record missing in search result even if the record like a
hit for query.
For Example:
Record in index is like- {"msg":"abc def"}
When I am searching for "abc" the record in not included in result but at
the same time when I am searching for "abc AND def" then record appear in
the result.
I am wondering whether there is some criteria in ElasticSearch based on
score or something else then ignore some records in result.
Please help.
Thanks,
Pulkit Agrawal
--
Hi,
I think you can get better help if you provide culr recreation.
Regards,
Lukas
On Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal pulkitdotcom@gmail.comwrote:
Hi
I am getting few record missing in search result even if the record like a
hit for query.For Example:
Record in index is like- {"msg":"abc def"}
When I am searching for "abc" the record in not included in result but at
the same time when I am searching for "abc AND def" then record appear in
the result.I am wondering whether there is some criteria in Elasticsearch based on
score or something else then ignore some records in result.Please help.
Thanks,
Pulkit Agrawal--
--
Sure Lukas
My first query:-
curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d
'{"query":{"filtered":
{"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR
(Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR
(Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR
(Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR
(Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR
(Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly
AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one
AND man AND one AND woman) OR (same AND gender AND
marriage))"}},{"field":{"UserGender":"F OR M OR
U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR
X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND
OR ITC OR MAT OR TEL OR UD OR UTL OR All OR
U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR
PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR
U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } ,
"filter": {"range":{"CreatedAt":
{"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} }
},"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["
bit.ly","rt","http","t.co","i","you","my","your""]}}}}'
*
*
It return count for term "great" as 207 but When I run the same query with
term "great" in AND the count changes to 322 but it should be 207. Second
query is a below.
curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d
'{"query":{"filtered":
{"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR
(Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR
(Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR
(Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR
(Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR
(Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly
AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one
AND man AND one AND woman) OR (same AND gender AND marriage)) AND
(great)"}},{"field":{"UserGender":"F
OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S
OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR
HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR
U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR
PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR
U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } ,
"filter": {"range":{"CreatedAt":
{"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} }
},"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["
bit.ly","rt","http","t.co","i","you","my","your""]}}}}'
Now term "great" count changes to 302.
*
*
Please suggest something.
*
*
Thanks,
*
*
On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček lukas.vlcek@gmail.com wrote:
Hi,
I think you can get better help if you provide culr recreation.
Elasticsearch Platform — Find real-time answers at scale | ElasticRegards,
LukasOn Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal pulkitdotcom@gmail.comwrote:
Hi
I am getting few record missing in search result even if the record like
a hit for query.For Example:
Record in index is like- {"msg":"abc def"}
When I am searching for "abc" the record in not included in result but at
the same time when I am searching for "abc AND def" then record appear in
the result.I am wondering whether there is some criteria in Elasticsearch based on
score or something else then ignore some records in result.Please help.
Thanks,
Pulkit Agrawal--
--
--
Hi,
Here is an example of a full curl recreation. https://gist.github.com/2965777
Please provide something similar. It will be easier to replay your case.
--
David 
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 17 sept. 2012 à 12:11, Pulkit Agrawal pulkitdotcom@gmail.com a écrit :
Sure Lukas
My first query:-
curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage))"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } , "filter": {"range":{"CreatedAt": {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["bit.ly","rt","http","t.co","i","you","my","your""]}}}}'
It return count for term "great" as 207 but When I run the same query with term "great" in AND the count changes to 322 but it should be 207. Second query is a below.
curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage)) AND (great)"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } , "filter": {"range":{"CreatedAt": {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["bit.ly","rt","http","t.co","i","you","my","your""]}}}}'
Now term "great" count changes to 302.
Please suggest something.
Thanks,
On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček lukas.vlcek@gmail.com wrote:
Hi,
I think you can get better help if you provide culr recreation.
http://www.elasticsearch.org/help/
Regards,
Lukas
On Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal pulkitdotcom@gmail.com wrote:
Hi
I am getting few record missing in search result even if the record like a hit for query.
For Example:
Record in index is like- {"msg":"abc def"}
When I am searching for "abc" the record in not included in result but at the same time when I am searching for "abc AND def" then record appear in the result.
I am wondering whether there is some criteria in ElasticSearch based on score or something else then ignore some records in result.
Please help.
--
--
--
Hope, it is more readable now
My first query:-
{
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{
"field": {
"MsgData": "((gayANDMarriage) OR
(MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR
(Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR
(Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR
(Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR
(Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly
AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one
AND man AND one AND woman) OR (same AND gender AND marriage))"
}
},
{
"field": {
"UserGender": "FORMORU"
}
},
{
"field": {
"DataSourceTypeID":
"BORTORNORDORFORVORSORRORX"
}
},
{
"field": {
"DataSourceIndustry":
"CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU"
}
},
{
"field": {
"DataSourceBusinessArea":
"CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU"
}
},
{
"field": {
"polarity": "2OR1"
}
}
],
"minimum_number_should_match": 6
}
},
"filter": {
"range": {
"CreatedAt": {
"from": "2012-9-10T00: 00: 00",
"to": "2012-9-15T23: 59: 59"
}
}
}
}
},
"facets": {
"MsgData": {
"terms": {
"field": "MsgData",
"size": 100,
"exclude": [
"bit.ly",
"rt",
"http",
"t.co",
"i",
"you",
"my",
"your"
]
}
}
}
}
*
*
It return count for term "great" as 207 but When I run the same query with
term "great" in AND the count changes to 322 but it should be 207. Second
query is a below.
{
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{
"field": {
"MsgData": " ((gayANDMarriage) OR
(MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR
(Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR
(Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR
(Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR
(Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly
AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one
AND man AND one AND woman) OR (same AND gender AND marriage)) AND (great)"
}
},
{
"field": {
"UserGender": "FORMORU"
}
},
{
"field": {
"DataSourceTypeID":
"BORTORNORDORFORVORSORRORX"
}
},
{
"field": {
"DataSourceIndustry":
"CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU"
}
},
{
"field": {
"DataSourceBusinessArea":
"CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU"
}
},
{
"field": {
"polarity": "2OR1"
}
}
],
"minimum_number_should_match": 6
}
},
"filter": {
"range": {
"CreatedAt": {
"from": "2012-9-10T00: 00: 00",
"to": "2012-9-15T23: 59: 59"
}
}
}
}
},
"facets": {
"MsgData": {
"terms": {
"field": "MsgData",
"size": 100,
"exclude": [
"bit.ly",
"rt",
"http",
"t.co",
"i",
"you",
"my",
"your"
]
}
}
}
}
Now term "great" count changes to 302.
*
*
Please suggest something.
*
*
Thanks,
*
*
On Mon, Sep 17, 2012 at 3:49 PM, David Pilato david@pilato.fr wrote:
Hi,
Here is an example of a full curl recreation.
Test case for multifield fields · GitHub
Please provide something similar. It will be easier to replay your case.--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocsLe 17 sept. 2012 à 12:11, Pulkit Agrawal pulkitdotcom@gmail.com a
écrit :Sure Lukas
My first query:-
curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d
'{"query":{"filtered":
{"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR
(Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR
(Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR
(Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR
(Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR
(Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly
AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one
AND man AND one AND woman) OR (same AND gender AND
marriage))"}},{"field":{"UserGender":"F OR M OR
U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR
X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND
OR ITC OR MAT OR TEL OR UD OR UTL OR All OR
U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR
PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR
U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } ,
"filter": {"range":{"CreatedAt":
{"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} }
},"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["
bit.ly","rt","http","t.co","i","you","my","your""]}}}}'
*
*
It return count for term "great" as 207 but When I run the same query
with term "great" in AND the count changes to 322 but it should be 207.
Second query is a below.curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d
'{"query":{"filtered":
{"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR
(Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR
(Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR
(Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR
(Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR
(Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly
AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one
AND man AND one AND woman) OR (same AND gender AND marriage)) *AND (great)
*"}},{"field":{"UserGender":"F OR M OR
U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR
X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND
OR ITC OR MAT OR TEL OR UD OR UTL OR All OR
U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR
PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR
U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } ,
"filter": {"range":{"CreatedAt":
{"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} }
},"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["
bit.ly","rt","http","t.co","i","you","my","your""]}}}}'Now term "great" count changes to 302.
*
*
Please suggest something.
*
*
Thanks,
*
*On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček lukas.vlcek@gmail.comwrote:
Hi,
I think you can get better help if you provide culr recreation.
Elasticsearch Platform — Find real-time answers at scale | ElasticRegards,
LukasOn Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal pulkitdotcom@gmail.comwrote:
Hi
I am getting few record missing in search result even if the record like
a hit for query.For Example:
Record in index is like- {"msg":"abc def"}
When I am searching for "abc" the record in not included in result but
at the same time when I am searching for "abc AND def" then record appear
in the result.I am wondering whether there is some criteria in Elasticsearch based on
score or something else then ignore some records in result.Please help.
Thanks,
Pulkit Agrawal--
--
--
--
--
I think you did not understand what Lukas and I are saying !
If I run your query on my ES instance. It won't work.
How can we help you if we are able to reproduce your use case?
We don't know your settings, your mappings, your docs...
--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 17 sept. 2012 à 12:30, Pulkit Agrawal pulkitdotcom@gmail.com a écrit :
Hope, it is more readable now
My first query:-
{
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{
"field": {
"MsgData": "((gayANDMarriage) OR (MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage))"
}
},
{
"field": {
"UserGender": "FORMORU"
}
},
{
"field": {
"DataSourceTypeID": "BORTORNORDORFORVORSORRORX"
}
},
{
"field": {
"DataSourceIndustry": "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU"
}
},
{
"field": {
"DataSourceBusinessArea": "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU"
}
},
{
"field": {
"polarity": "2OR1"
}
}
],
"minimum_number_should_match": 6
}
},
"filter": {
"range": {
"CreatedAt": {
"from": "2012-9-10T00: 00: 00",
"to": "2012-9-15T23: 59: 59"
}
}
}
}
},
"facets": {
"MsgData": {
"terms": {
"field": "MsgData",
"size": 100,
"exclude": [
"bit.ly",
"rt",
"http",
"t.co",
"i",
"you",
"my",
"your"
]
}
}
}
}
It return count for term "great" as 207 but When I run the same query with term "great" in AND the count changes to 322 but it should be 207. Second query is a below.
{
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{
"field": {
"MsgData": " ((gayANDMarriage) OR (MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage)) AND (great)"
}
},
{
"field": {
"UserGender": "FORMORU"
}
},
{
"field": {
"DataSourceTypeID": "BORTORNORDORFORVORSORRORX"
}
},
{
"field": {
"DataSourceIndustry": "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU"
}
},
{
"field": {
"DataSourceBusinessArea": "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU"
}
},
{
"field": {
"polarity": "2OR1"
}
}
],
"minimum_number_should_match": 6
}
},
"filter": {
"range": {
"CreatedAt": {
"from": "2012-9-10T00: 00: 00",
"to": "2012-9-15T23: 59: 59"
}
}
}
}
},
"facets": {
"MsgData": {
"terms": {
"field": "MsgData",
"size": 100,
"exclude": [
"bit.ly",
"rt",
"http",
"t.co",
"i",
"you",
"my",
"your"
]
}
}
}
}
Now term "great" count changes to 302.
Please suggest something.
Thanks,
On Mon, Sep 17, 2012 at 3:49 PM, David Pilato david@pilato.fr wrote:
Hi,
Here is an example of a full curl recreation. https://gist.github.com/2965777
Please provide something similar. It will be easier to replay your case.
--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 17 sept. 2012 à 12:11, Pulkit Agrawal pulkitdotcom@gmail.com a écrit :
Sure Lukas
My first query:-
curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage))"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } , "filter": {"range":{"CreatedAt": {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["bit.ly","rt","http","t.co","i","you","my","your""]}}}}'
It return count for term "great" as 207 but When I run the same query with term "great" in AND the count changes to 322 but it should be 207. Second query is a below.
curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage)) AND (great)"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } , "filter": {"range":{"CreatedAt": {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["bit.ly","rt","http","t.co","i","you","my","your""]}}}}'
Now term "great" count changes to 302.
Please suggest something.
Thanks,
On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček lukas.vlcek@gmail.com wrote:
Hi,
I think you can get better help if you provide culr recreation.
http://www.elasticsearch.org/help/
Regards,
Lukas
On Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal pulkitdotcom@gmail.com wrote:
Hi
I am getting few record missing in search result even if the record like a hit for query.
For Example:
Record in index is like- {"msg":"abc def"}
When I am searching for "abc" the record in not included in result but at the same time when I am searching for "abc AND def" then record appear in the result.
I am wondering whether there is some criteria in ElasticSearch based on score or something else then ignore some records in result.
Please help.
--
--
--
--
--
here is my mapping of index
curl -XPUT 'localhost:9200/inn/innt/_mapping' -d '{
"innt": {
"properties": {
"FBLikeCount": {
"type": "integer"
},
"FBShareCount": {
"type": "integer"
},
"FBRecommendCount": {
"type": "integer"
},
"FBTotalCount": {
"type": "integer"
},
"FBClickCount": {
"type": "integer"
},
"TWShareCount": {
"type": "integer"
},
"NoOfComments": {
"type": "integer"
},
"VideoWatchCount": {
"type": "integer"
},
"AuthorNumberOfResult1": {
"type": "integer"
},
"polarity": {
"type": "multi_field",
"fields": {
"polarity": {
"type": "string"
},
"polarityint": {
"type": "integer"
}
}
}
}
}
}}'
Here is my sample document:
{
"_id": "245745919428014081",
"_rev": "1-e863bfb49a313b61f1a07cf3224dad49",
"SystemCreatedAt": "2012-09-12T05:13:32",
"MsgData": "I shoulda taken my medicine. Caught a bad case of feelings.",
"IsTweet": "Y",
"CreatedAt": "2012-9-12T04:49:27",
"Retweet": "N",
"SystemModifiedAt": "2012-09-12T05:13:32",
"ReplyTweetID": "",
"ReplyUserID": "",
"IsEnglish": "Y",
"polarity": "2",
"MessageEfluencerIndex": 91409.825,
"BusinessAreaConfiguration": "UD",
"Feature": "",
"LifeStyle": " HLT",
"Source": "Echofon",
"AuthorLocationKnow": "Y",
"AuthorLocationContinentCode": "NORTHAMERICA",
"AuthorLocationCountryCode": "US",
"AuthorLocationStateCode": " WY",
"GeoEnabled": "N",
"PlaceFullName": "",
"PlaceName": "",
"TweetContry": "",
"TweetCountryCode": "",
"TweetLatitude": "",
"TweetLongigttude": "",
"UserID": "23914462",
"UserName": "Proto_Zsay",
"ToUserID": "",
"ToUserName": "",
"IsPopular": "N",
"UserGender": "M",
"Name": "Pres. Chill Clinton",
"Location": "Byron Bay, NSW",
"UserEfluncerIndex": 18281.765,
"AuthorLocationRegionCode": "W",
"RetweetCount": "0",
"DataSourceTypeID": "T",
"DataSourceIndustry": "All",
"DataSourceBusinessArea": "All",
"DataSourceCoverageArea": "All"
}
Other than that there is no change in setting. I am using default
Elasticsearch setting of 0.18.7.
Let me know, if you are looking for any other information. I am happy to
provide the same.
Thanks,
On Mon, Sep 17, 2012 at 4:16 PM, David Pilato david@pilato.fr wrote:
I think you did not understand what Lukas and I are saying !
If I run your query on my ES instance. It won't work.
How can we help you if we are able to reproduce your use case?We don't know your settings, your mappings, your docs...
--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocsLe 17 sept. 2012 à 12:30, Pulkit Agrawal pulkitdotcom@gmail.com a
écrit :Hope, it is more readable now
My first query:-
{
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{
"field": {
"MsgData": "((gayANDMarriage) OR
(MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR
(Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR
(Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR
(Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR
(Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly
AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one
AND man AND one AND woman) OR (same AND gender AND marriage))"
}
},
{
"field": {
"UserGender": "FORMORU"
}
},
{
"field": {
"DataSourceTypeID":
"BORTORNORDORFORVORSORRORX"
}
},
{
"field": {
"DataSourceIndustry":
"CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU"
}
},
{
"field": {
"DataSourceBusinessArea":
"CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU"
}
},
{
"field": {
"polarity": "2OR1"
}
}
],
"minimum_number_should_match": 6
}
},
"filter": {
"range": {
"CreatedAt": {
"from": "2012-9-10T00: 00: 00",
"to": "2012-9-15T23: 59: 59"
}
}
}
}
},
"facets": {
"MsgData": {
"terms": {
"field": "MsgData",
"size": 100,
"exclude": [
"bit.ly",
"rt",
"http",
"t.co",
"i",
"you",
"my",
"your"
]
}
}
}
}
*
*
It return count for term "great" as 207 but When I run the same query
with term "great" in AND the count changes to 322 but it should be 207.
Second query is a below.{
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{
"field": {
"MsgData": " ((gayANDMarriage) OR
(MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR
(Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR
(Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR
(Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR
(Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly
AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one
AND man AND one AND woman) OR (same AND gender AND marriage)) *AND (great)
*"
}
},
{
"field": {
"UserGender": "FORMORU"
}
},
{
"field": {
"DataSourceTypeID":
"BORTORNORDORFORVORSORRORX"
}
},
{
"field": {
"DataSourceIndustry":
"CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU"
}
},
{
"field": {
"DataSourceBusinessArea":
"CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU"
}
},
{
"field": {
"polarity": "2OR1"
}
}
],
"minimum_number_should_match": 6
}
},
"filter": {
"range": {
"CreatedAt": {
"from": "2012-9-10T00: 00: 00",
"to": "2012-9-15T23: 59: 59"
}
}
}
}
},
"facets": {
"MsgData": {
"terms": {
"field": "MsgData",
"size": 100,
"exclude": [
"bit.ly",
"rt",
"http",
"t.co",
"i",
"you",
"my",
"your"
]
}
}
}
}Now term "great" count changes to 302.
*
*
Please suggest something.
*
*
Thanks,
*
*On Mon, Sep 17, 2012 at 3:49 PM, David Pilato david@pilato.fr wrote:
Hi,
Here is an example of a full curl recreation.
Test case for multifield fields · GitHub
Please provide something similar. It will be easier to replay your case.--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocsLe 17 sept. 2012 à 12:11, Pulkit Agrawal pulkitdotcom@gmail.com a
écrit :Sure Lukas
My first query:-
curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d
'{"query":{"filtered":
{"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR
(Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR
(Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR
(Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR
(Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR
(Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly
AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one
AND man AND one AND woman) OR (same AND gender AND
marriage))"}},{"field":{"UserGender":"F OR M OR
U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR
X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND
OR ITC OR MAT OR TEL OR UD OR UTL OR All OR
U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR
PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR
U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } ,
"filter": {"range":{"CreatedAt":
{"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} }
},"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["
bit.ly","rt","http","t.co","i","you","my","your""]}}}}'
*
*
It return count for term "great" as 207 but When I run the same query
with term "great" in AND the count changes to 322 but it should be 207.
Second query is a below.curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d
'{"query":{"filtered":
{"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR
(Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR
(Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR
(Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR
(Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR
(Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly
AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one
AND man AND one AND woman) OR (same AND gender AND marriage)) AND
(great)"}},{"field":{"UserGender":"F OR M OR
U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR
X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND
OR ITC OR MAT OR TEL OR UD OR UTL OR All OR
U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR
PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR
U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } ,
"filter": {"range":{"CreatedAt":
{"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} }
},"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["
bit.ly","rt","http","t.co","i","you","my","your""]}}}}'Now term "great" count changes to 302.
*
*
Please suggest something.
*
*
Thanks,
*
*On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček lukas.vlcek@gmail.comwrote:
Hi,
I think you can get better help if you provide culr recreation.
Elasticsearch Platform — Find real-time answers at scale | ElasticRegards,
LukasOn Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal <pulkitdotcom@gmail.com
wrote:
Hi
I am getting few record missing in search result even if the record
like a hit for query.For Example:
Record in index is like- {"msg":"abc def"}
When I am searching for "abc" the record in not included in result but
at the same time when I am searching for "abc AND def" then record appear
in the result.I am wondering whether there is some criteria in Elasticsearch based on
score or something else then ignore some records in result.Please help.
Thanks,
Pulkit Agrawal--
--
--
--
--
--
--
Hi Pulkit
Have a look at Elasticsearch Platform — Find real-time answers at scale | Elastic
It explains how to provide code samples to make it easy for others to
debug problems. If we can't copy and paste something, then we are
unlikely to try it.
Other than that there is no change in setting. I am using default
Elasticsearch setting of 0.18.7.
Also, I would upgrade elasticsearch. A number of bugs have been fixed
since 0.18.7
clint
Let me know, if you are looking for any other information. I am happy
to provide the same.Thanks,
On Mon, Sep 17, 2012 at 4:16 PM, David Pilato david@pilato.fr wrote:
I think you did not understand what Lukas and I are saying !If I run your query on my ES instance. It won't work. How can we help you if we are able to reproduce your use case? We don't know your settings, your mappings, your docs... -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 17 sept. 2012 à 12:30, Pulkit Agrawal <pulkitdotcom@gmail.com> a écrit : Hope, it is more readable now My first query:- { "query": { "filtered": { "query": { "bool": { "should": [ { "field": { "MsgData": "((gayANDMarriage) OR (MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage))" } }, { "field": { "UserGender": "FORMORU" } }, { "field": { "DataSourceTypeID": "BORTORNORDORFORVORSORRORX" } }, { "field": { "DataSourceIndustry": "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" } }, { "field": { "DataSourceBusinessArea": "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU" } }, { "field": { "polarity": "2OR1" } } ], "minimum_number_should_match": 6 } }, "filter": { "range": { "CreatedAt": { "from": "2012-9-10T00: 00: 00", "to": "2012-9-15T23: 59: 59" } } } } }, "facets": { "MsgData": { "terms": { "field": "MsgData", "size": 100, "exclude": [ "bit.ly", "rt", "http", "t.co", "i", "you", "my", "your" ] } } } } It return count for term "great" as 207 but When I run the same query with term "great" in AND the count changes to 322 but it should be 207. Second query is a below. { "query": { "filtered": { "query": { "bool": { "should": [ { "field": { "MsgData": " ((gayANDMarriage) OR (MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage)) AND (great)" } }, { "field": { "UserGender": "FORMORU" } }, { "field": { "DataSourceTypeID": "BORTORNORDORFORVORSORRORX" } }, { "field": { "DataSourceIndustry": "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" } }, { "field": { "DataSourceBusinessArea": "CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU" } }, { "field": { "polarity": "2OR1" } } ], "minimum_number_should_match": 6 } }, "filter": { "range": { "CreatedAt": { "from": "2012-9-10T00: 00: 00", "to": "2012-9-15T23: 59: 59" } } } } }, "facets": { "MsgData": { "terms": { "field": "MsgData", "size": 100, "exclude": [ "bit.ly", "rt", "http", "t.co", "i", "you", "my", "your" ] } } } } Now term "great" count changes to 302. Please suggest something. Thanks, On Mon, Sep 17, 2012 at 3:49 PM, David Pilato <david@pilato.fr> wrote: Hi, Here is an example of a full curl recreation. https://gist.github.com/2965777 Please provide something similar. It will be easier to replay your case. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 17 sept. 2012 à 12:11, Pulkit Agrawal <pulkitdotcom@gmail.com> a écrit : Sure Lukas My first query:- curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage))"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } , "filter": {"range":{"CreatedAt": {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["bit.ly","rt","http","t.co","i","you","my","your""]}}}}' It return count for term "great" as 207 but When I run the same query with term "great" in AND the count changes to 322 but it should be 207. Second query is a below. curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage)) AND (great)"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } , "filter": {"range":{"CreatedAt": {"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} } },"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["bit.ly","rt","http","t.co","i","you","my","your""]}}}}' Now term "great" count changes to 302. Please suggest something. Thanks, On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček <lukas.vlcek@gmail.com> wrote: Hi, I think you can get better help if you provide culr recreation. http://www.elasticsearch.org/help/ Regards, Lukas On Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal <pulkitdotcom@gmail.com> wrote: Hi I am getting few record missing in search result even if the record like a hit for query. For Example: Record in index is like- {"msg":"abc def"} When I am searching for "abc" the record in not included in result but at the same time when I am searching for "abc AND def" then record appear in the result. I am wondering whether there is some criteria in ElasticSearch based on score or something else then ignore some records in result. Please help. Thanks, Pulkit Agrawal -- -- -- -- --
--
Thanks.
I found same issue here
Now, can anybody help me?
I can not use single shard. What else I can do?
On Mon, Sep 17, 2012 at 4:33 PM, Clinton Gormley clint@traveljury.comwrote:
Hi Pulkit
Have a look at Elasticsearch Platform — Find real-time answers at scale | Elastic
It explains how to provide code samples to make it easy for others to
debug problems. If we can't copy and paste something, then we are
unlikely to try it.Other than that there is no change in setting. I am using default
Elasticsearch setting of 0.18.7.Also, I would upgrade elasticsearch. A number of bugs have been fixed
since 0.18.7clint
Let me know, if you are looking for any other information. I am happy
to provide the same.Thanks,
On Mon, Sep 17, 2012 at 4:16 PM, David Pilato david@pilato.fr wrote:
I think you did not understand what Lukas and I are saying !If I run your query on my ES instance. It won't work. How can we help you if we are able to reproduce your use case? We don't know your settings, your mappings, your docs... -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 17 sept. 2012 à 12:30, Pulkit Agrawal <pulkitdotcom@gmail.com> a écrit : Hope, it is more readable now My first query:- { "query": { "filtered": { "query": { "bool": { "should": [ { "field": { "MsgData": "((gayANDMarriage) OR (MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage))" } }, { "field": { "UserGender": "FORMORU" } }, { "field": { "DataSourceTypeID": "BORTORNORDORFORVORSORRORX" } }, { "field": { "DataSourceIndustry": "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" } }, { "field": { "DataSourceBusinessArea":"CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU"
} }, { "field": { "polarity": "2OR1" } } ], "minimum_number_should_match": 6 } }, "filter": { "range": { "CreatedAt": { "from": "2012-9-10T00: 00: 00", "to": "2012-9-15T23: 59: 59" } } } } }, "facets": { "MsgData": { "terms": { "field": "MsgData", "size": 100, "exclude": [ "bit.ly", "rt", "http", "t.co", "i", "you", "my", "your" ] } } } } It return count for term "great" as 207 but When I run the same query with term "great" in AND the count changes to 322 but it should be 207. Second query is a below. { "query": { "filtered": { "query": { "bool": { "should": [ { "field": { "MsgData": " ((gayANDMarriage) OR (MarriageANDequality) OR (LGBTANDmarriage) OR (SamesexANDmarriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND ) OR (Marriage AND quality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage)) AND (great)" } }, { "field": { "UserGender": "FORMORU" } }, { "field": { "DataSourceTypeID": "BORTORNORDORFORVORSORRORX" } }, { "field": { "DataSourceIndustry": "CDCORCSPOREGYORFNSORHLCORINDORITCORMATORTELORUDORUTLORAllORU" } }, { "field": { "DataSourceBusinessArea":"CRSORCUSORMKIORPRCORPFORPTLORCGNORFNIORLGIORMAOROPEORPLTORTNTORUDORAllORU"
} }, { "field": { "polarity": "2OR1" } } ], "minimum_number_should_match": 6 } }, "filter": { "range": { "CreatedAt": { "from": "2012-9-10T00: 00: 00", "to": "2012-9-15T23: 59: 59" } } } } }, "facets": { "MsgData": { "terms": { "field": "MsgData", "size": 100, "exclude": [ "bit.ly", "rt", "http", "t.co", "i", "you", "my", "your" ] } } } } Now term "great" count changes to 302. Please suggest something. Thanks, On Mon, Sep 17, 2012 at 3:49 PM, David Pilato <david@pilato.fr> wrote: Hi, Here is an example of a full curl recreation. https://gist.github.com/2965777 Please provide something similar. It will be easier to replay your case. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 17 sept. 2012 à 12:11, Pulkit Agrawal <pulkitdotcom@gmail.com> a écrit : Sure Lukas My first query:- curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage))"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } , "filter": {"range":{"CreatedAt":{"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} }
},"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["
bit.ly","rt","http","t.co","i","you","my","your""]}}}}'It return count for term "great" as 207 but When I run the same query with term "great" in AND the count changes to 322 but it should be 207. Second query is a below. curl -XGET 'localhost:9200XXX/YYYt/_search?size=0' -d '{"query":{"filtered": {"query":{"bool":{"should":[{"field":{"MsgData":"((gay AND Marriage) OR (Marriage AND equality) OR (LGBT AND marriage) OR (Samesex AND marriage) OR (Prop8) OR (Samesex) OR (Marriageequality AND New AND Jersey) OR (Marriageequality AND Minnesota) OR (Marriageequality AND New AND YORk) OR (Marriageequality AND MarylAND) OR (Marriage AND Equality AND Act) OR (Connie AND Ress) OR (Michael AND Thurber) OR (Jim AND DeLaHunt) OR (Molly AND McKay) OR (Davina AND Kotulski) OR (Marriageequality AND ban) OR (one AND man AND one AND woman) OR (same AND gender AND marriage)) AND (great)"}},{"field":{"UserGender":"F OR M OR U"}},{"field":{"DataSourceTypeID":"B OR T OR N OR D OR F OR V OR S OR R OR X"}},{"field":{"DataSourceIndustry":"CDC OR CSP OR EGY OR FNS OR HLC OR IND OR ITC OR MAT OR TEL OR UD OR UTL OR All OR U"}},{"field":{"DataSourceBusinessArea":"CRS OR CUS OR MKI OR PRC OR PF OR PTL OR CGN OR FNI OR LGI OR MA OR OPE OR PLT OR TNT OR UD OR All OR U"}},{"field":{"polarity":"2 OR 1"}}],"minimum_number_should_match":6} } , "filter": {"range":{"CreatedAt":{"from":"2012-9-10T00:00:00","to":"2012-9-15T23:59:59"}}} }
},"facets":{"MsgData":{"terms":{"field":"MsgData","size":100,"exclude":["
bit.ly","rt","http","t.co","i","you","my","your""]}}}}'Now term "great" count changes to 302. Please suggest something. Thanks, On Sun, Sep 16, 2012 at 12:13 AM, Lukáš Vlček <lukas.vlcek@gmail.com> wrote: Hi, I think you can get better help if you provide culr recreation. http://www.elasticsearch.org/help/ Regards, Lukas On Sat, Sep 15, 2012 at 10:19 AM, Pulkit Agrawal <pulkitdotcom@gmail.com> wrote: Hi I am getting few record missing in search result even if the record like a hit for query. For Example: Record in index is like- {"msg":"abc def"} When I am searching for "abc" the record in not included in result but at the same time when I am searching for "abc AND def" then record appear in the result. I am wondering whether there is some criteria in ElasticSearch based on score or something else then ignore some records in result. Please help. Thanks, Pulkit Agrawal -- -- -- -- ----
--
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.
