Compound Must Matches Not Working for Me

Douglas_McQuilken · August 27, 2018, 6:39pm

Everything works fine until I insert a second match statement with a MUST, see below.
NOTE: Either individual MUST statement works as expected but in combination...
Parses correctly BUT always returns zero hits.
Assistance (rather than more frustration) would be greatly appreciated

##############################################################

{"size": 1000,
"query" : {
"bool" : {
"minimum_should_match": 1,
"must" : [{"match": { "source": { "query": "", "fuzziness": "0","operator": "and"}}},
{"match": { "today": {"query": "<2018-238>"}}}],
"should" : [
{"match": { "description": { "query": "<- brass - ->", "fuzziness": "0","operator": "and"}}},
{"match": { "url": { "query": "<- brass - ->", "fuzziness": "0","operator": "and"}}},
{"match": {"description": { "query": "<*>","fuzziness": "0","operator": "and"}}}
]
,"must_not":[
{"match":{"description": "ss"}}
]
}
}
}

abdon · August 28, 2018, 12:06pm

Are you sure there are documents that match both of the must clauses? You can think of multiple must clauses as an AND - a document must match ALL of the clauses to be returned as a hit.

Maybe you can share an example of a document that matches each individual must clause, but not the combined two?

Douglas_McQuilken · August 28, 2018, 4:40pm

Thanks, upon reviewing my post I realized that I mangled the example

Following is:
(1) a document (2) After that is (3) Lastly, is the ######################### {"_index":"bupstest9","_t ######################### {"size": 1000,
"query" : {
"bool" : {
"minimum_should_match": 1,
"must" : [{"match": { "source": {"match": { "today": "should" : [
{"match": { "description": {"match": { "url": {"match": { "description": {"match": { "url": {"match": { "description": {"match": { "url": {"match": { "description": {"match": { "url": {"match": {"description": ]
,"must_not":[
{"match":{"description": "ss"}}
]
}
}
}
######################### {"size": 1000,
"query" : {
"bool" : {
"should" : [
{"match": { "description": {"match": { "url": {"match": { "description": {"match": { "url": {"match": { "description": {"match": { "url": {"match": { "description": {"match": { "url": {"match": {"description": ]
,"must_not":[
{"match":{"description": "ss"}}
]
}
}
} which matches all criteria (I located this independently)
the actual query which parsed but provided 0 hits.
same DSL query without the MUST which results in this document (among hundreds more)
#################################
ype":"listings","_id":"https:~~outerbanks.craigslist.org~bpo~d~30-nibral-4-blade-class-1~6652278553.html","_score":9.197452,"_source":{"url": "https://outerbanks.craigslist.org/bpo/d/30-nibral-4-blade-class-1/6652278553.html","description": "@@30" nibral 4-BLADE CLASS 1 tuned props factory-direcT PRICES@@ - ","site": "Craigslist_outer_banks","city": "outer_banks","state": "NorthCarolina","country": "US","source": "Craigslist","bupsid": "9227198270","year": "2018","day": "239","today": "2018-239"}}
################################
{ "query": "", "fuzziness": "0","operator": "and"}}},
{"query": "<2018-239>"}}}],
{ "query": "<- nibral - prop>", "fuzziness": "0","operator": "and"}}},
{ "query": "<- nibral - prop>", "fuzziness": "0","operator": "and"}}},
{ "query": "<- nibral - propeller >", "fuzziness": "0","operator": "and"}}},
{ "query": "<- nibral - propeller >", "fuzziness": "0","operator": "and"}}},
{ "query": "<- ni-bral - prop>", "fuzziness": "0","operator": "and"}}},
{ "query": "<- ni-bral - prop>", "fuzziness": "0","operator": "and"}}},
{ "query": "<- ni-bral - propeller >", "fuzziness": "0","operator": "and"}}},
{ "query": "<- ni-bral - propeller >", "fuzziness": "0","operator": "and"}}},
{ "query": "<>","fuzziness": "0","operator": "and"}}}
###########################
{ "query": "<- nibral - prop>", "fuzziness": "0","operator": "and"}}},
{ "query": "<- nibral - prop>", "fuzziness": "0","operator": "and"}}},
{ "query": "<- nibral - propeller >", "fuzziness": "0","operator": "and"}}},
{ "query": "<- nibral - propeller >", "fuzziness": "0","operator": "and"}}},
{ "query": "<- ni-bral - prop>", "fuzziness": "0","operator": "and"}}},
{ "query": "<- ni-bral - prop>", "fuzziness": "0","operator": "and"}}},
{ "query": "<- ni-bral - propeller >", "fuzziness": "0","operator": "and"}}},
{ "query": "<- ni-bral - propeller >", "fuzziness": "0","operator": "and"}}},
{ "query": "<>","fuzziness": "0","operator": "and"}}}

abdon · August 28, 2018, 5:06pm

I'm not sure what the first match clause is supposed to do. Why search for an empty string with fuzziness set to 0 and the operator set to and? If I execute that query agains the document that you posted it does not match the document. This clause is the cause of the entire query not matching your document.

GET bupstest9/_search
{
  "query": {
    "match": {
      "source": {
        "query": "",
        "fuzziness": "0",
        "operator": "and"
      }
    }
  }
}

PS Please format the code that you post here, as described here. It makes it much easier to read.

Douglas_McQuilken · August 28, 2018, 5:37pm

It appears that you are working from the first example I posted which has a typo
Please refer to my subsequent posting which has a target document meeting the criteria as well as complete DSL examples.

In short, what I am attempting to accomplish is to have a MUST containing multiple match/queries. Therse would be ANDed together to filter the document set. Then I intended to use a SHOULD to process within this subset.

The query worked properly with a single match/query within the MUST.
Once I added a second match/query, it parsed but failed to find known documents.
I couldn't find exact examples to work from - so I paralleled the SHOULD construct......

Thanks for your continuing interest

abdon · August 28, 2018, 5:47pm

Your second example (for the query that has 0 hits) has the following must clause:

"must" : [{"match": { "source": { "query": "", "fuzziness": "0","operator": "and"}}}, {"match": { "today": {"query": "<2018-239>"}}}]

Note that one of the two queries inside of that must clause, is the match clause that searches for an empty string with fuzziness 0 and operator AND. This query is your problem query.

Douglas_McQuilken · August 28, 2018, 6:18pm

AHA - As we used to say the northeast "light dawns on Marblehead"
NOTE; this is a town on the coast of Massachusetts

In any case, I failed to follow the thread insert code instructions
Thus, my query word is being ignored within this thread.
It has the same syntax as the second match/query word, i.e. "LTCraigslistGT"

abdon · August 28, 2018, 6:42pm

Ha, believe it or not, but I visited Marblehead last summer. It is a lovely town.

So, you're saying the must clause looks like this?

      "must": [
        {
          "match": {
            "source": {
              "query": "<Craigslist>",
              "fuzziness": "0",
              "operator": "and"
            }
          }
        },
        {
          "match": {
            "today": {
              "query": "<2018-239>"
            }
          }
        }
      ]

Let me ask you: why the angle brackets in your queries? Those should not be there. Anyway, they should not influence the search results normally.

Would you mind sharing the settings and the mappings of the index? Just do a

GET bupstest9

and post the output. If it's too large, maybe you can create a gist and share a link to that.

Douglas_McQuilken · August 28, 2018, 8:37pm

I believe the angle brackets are left over from ES 2.x

{"bupstest9":{"aliases":{},"mappings":{"listings":{"properties":{"bupsid":{"type":"text"},"city":{"type":"text"},"country":{"type":"text"},"day":{"type":"text"},"description":{"type":"text","analyzer":"snowball"},"site":{"type":"text"},"source":{"type":"text"},"state":{"type":"text"},"today":{"type":"text"},"url":{"type":"text","analyzer":"snowball"},"year":{"type":"text"}}}},"settings":{"index":{"creation_date":"1531007073465","number_of_shards":"5","number_of_replicas":"1","uuid":"TBJYwFCMRb-iM9xkqdMp-g","version":{"created":"6030099"},"provided_name":"bupstest9"}}}}

Douglas_McQuilken · August 30, 2018, 12:11pm

Finally, I have it working.
The problem was in my (mis)use of the "operator" construct within the queries, within the matches, within the must...
Thanks for your assistance.

system · September 27, 2018, 12:11pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Must and should match doesn't return should matches Elasticsearch	2	326	March 28, 2023
Elasticsearch \| Should query combining multiple clauses Elasticsearch	1	444	February 1, 2019
Must and should query together Elasticsearch	2	664	July 5, 2017
Bool query with a must and 2 match queries Elasticsearch	1	354	July 13, 2020
Minimum should match is not working Elasticsearch	5	2090	September 2, 2019

Compound Must Matches Not Working for Me

Related topics