Compound Must Matches Not Working for Me


#1

Everything works fine until I insert a second match statement with a MUST, see below.
NOTE: Either individual MUST statement works as expected but in combination...
Parses correctly BUT always returns zero hits.
Assistance (rather than more frustration) would be greatly appreciated :wink:

##############################################################

{"size": 1000,
"query" : {
"bool" : {
"minimum_should_match": 1,
"must" : [{"match": { "source": { "query": "", "fuzziness": "0","operator": "and"}}},
{"match": { "today": {"query": "<2018-238>"}}}],
"should" : [
{"match": { "description": { "query": "<- brass - ->", "fuzziness": "0","operator": "and"}}},
{"match": { "url": { "query": "<- brass - ->", "fuzziness": "0","operator": "and"}}},
{"match": {"description": { "query": "<*>","fuzziness": "0","operator": "and"}}}
]
,"must_not":[
{"match":{"description": "ss"}}
]
}
}
}


(Abdon Pijpelink) #2

Are you sure there are documents that match both of the must clauses? You can think of multiple must clauses as an AND - a document must match ALL of the clauses to be returned as a hit.

Maybe you can share an example of a document that matches each individual must clause, but not the combined two?


#3

Thanks, upon reviewing my post I realized that I mangled the example :frowning:

Following is:
(1) a document which matches all criteria (I located this independently)
(2) After that is the actual query which parsed but provided 0 hits.
(3) Lastly, is the same DSL query without the MUST which results in this document (among hundreds more)
##########################################################
{"_index":"bupstest9","_type":"listings","_id":"https:~~outerbanks.craigslist.org~bpo~d~30-nibral-4-blade-class-1~6652278553.html","_score":9.197452,"_source":{"url": "https://outerbanks.craigslist.org/bpo/d/30-nibral-4-blade-class-1/6652278553.html","description": "@@30" nibral 4-BLADE CLASS 1 tuned props factory-direcT PRICES@@ - ","site": "Craigslist_outer_banks","city": "outer_banks","state": "NorthCarolina","country": "US","source": "Craigslist","bupsid": "9227198270","year": "2018","day": "239","today": "2018-239"}}
#########################################################
{"size": 1000,
"query" : {
"bool" : {
"minimum_should_match": 1,
"must" : [{"match": { "source": { "query": "", "fuzziness": "0","operator": "and"}}},
{"match": { "today": {"query": "<2018-239>"}}}],
"should" : [
{"match": { "description": { "query": "<- nibral - prop>", "fuzziness": "0","operator": "and"}}},
{"match": { "url": { "query": "<- nibral - prop>", "fuzziness": "0","operator": "and"}}},
{"match": { "description": { "query": "<- nibral - propeller >", "fuzziness": "0","operator": "and"}}},
{"match": { "url": { "query": "<- nibral - propeller >", "fuzziness": "0","operator": "and"}}},
{"match": { "description": { "query": "<- ni-bral - prop>", "fuzziness": "0","operator": "and"}}},
{"match": { "url": { "query": "<- ni-bral - prop>", "fuzziness": "0","operator": "and"}}},
{"match": { "description": { "query": "<- ni-bral - propeller >", "fuzziness": "0","operator": "and"}}},
{"match": { "url": { "query": "<- ni-bral - propeller >", "fuzziness": "0","operator": "and"}}},
{"match": {"description": { "query": "<>","fuzziness": "0","operator": "and"}}}
]
,"must_not":[
{"match":{"description": "ss"}}
]
}
}
}
####################################################
{"size": 1000,
"query" : {
"bool" : {
"should" : [
{"match": { "description": { "query": "<- nibral - prop>", "fuzziness": "0","operator": "and"}}},
{"match": { "url": { "query": "<- nibral - prop>", "fuzziness": "0","operator": "and"}}},
{"match": { "description": { "query": "<- nibral - propeller >", "fuzziness": "0","operator": "and"}}},
{"match": { "url": { "query": "<- nibral - propeller >", "fuzziness": "0","operator": "and"}}},
{"match": { "description": { "query": "<- ni-bral - prop>", "fuzziness": "0","operator": "and"}}},
{"match": { "url": { "query": "<- ni-bral - prop>", "fuzziness": "0","operator": "and"}}},
{"match": { "description": { "query": "<- ni-bral - propeller >", "fuzziness": "0","operator": "and"}}},
{"match": { "url": { "query": "<- ni-bral - propeller >", "fuzziness": "0","operator": "and"}}},
{"match": {"description": { "query": "<
>","fuzziness": "0","operator": "and"}}}
]
,"must_not":[
{"match":{"description": "ss"}}
]
}
}
}


(Abdon Pijpelink) #4

I'm not sure what the first match clause is supposed to do. Why search for an empty string with fuzziness set to 0 and the operator set to and? If I execute that query agains the document that you posted it does not match the document. This clause is the cause of the entire query not matching your document.

GET bupstest9/_search
{
  "query": {
    "match": {
      "source": {
        "query": "",
        "fuzziness": "0",
        "operator": "and"
      }
    }
  }
}

PS Please format the code that you post here, as described here. It makes it much easier to read. :slight_smile:


#5

It appears that you are working from the first example I posted which has a typo :frowning:
Please refer to my subsequent posting which has a target document meeting the criteria as well as complete DSL examples.

In short, what I am attempting to accomplish is to have a MUST containing multiple match/queries. Therse would be ANDed together to filter the document set. Then I intended to use a SHOULD to process within this subset.

The query worked properly with a single match/query within the MUST.
Once I added a second match/query, it parsed but failed to find known documents.
I couldn't find exact examples to work from - so I paralleled the SHOULD construct......

Thanks for your continuing interest


(Abdon Pijpelink) #6

Your second example (for the query that has 0 hits) has the following must clause:

"must" : [{"match": { "source": { "query": "", "fuzziness": "0","operator": "and"}}}, {"match": { "today": {"query": "&lt;2018-239&gt;"}}}]

Note that one of the two queries inside of that must clause, is the match clause that searches for an empty string with fuzziness 0 and operator AND. This query is your problem query.


#7

AHA - As we used to say the northeast "light dawns on Marblehead"
NOTE; this is a town on the coast of Massachusetts

In any case, I failed to follow the thread insert code instructions :frowning:
Thus, my query word is being ignored within this thread.
It has the same syntax as the second match/query word, i.e. "LTCraigslistGT"


(Abdon Pijpelink) #8

Ha, believe it or not, but I visited Marblehead last summer. It is a lovely town.

So, you're saying the must clause looks like this?

      "must": [
        {
          "match": {
            "source": {
              "query": "<Craigslist>",
              "fuzziness": "0",
              "operator": "and"
            }
          }
        },
        {
          "match": {
            "today": {
              "query": "<2018-239>"
            }
          }
        }
      ]

Let me ask you: why the angle brackets in your queries? Those should not be there. Anyway, they should not influence the search results normally.

Would you mind sharing the settings and the mappings of the index? Just do a

GET bupstest9

and post the output. If it's too large, maybe you can create a gist and share a link to that.


#9

I believe the angle brackets are left over from ES 2.x

{"bupstest9":{"aliases":{},"mappings":{"listings":{"properties":{"bupsid":{"type":"text"},"city":{"type":"text"},"country":{"type":"text"},"day":{"type":"text"},"description":{"type":"text","analyzer":"snowball"},"site":{"type":"text"},"source":{"type":"text"},"state":{"type":"text"},"today":{"type":"text"},"url":{"type":"text","analyzer":"snowball"},"year":{"type":"text"}}}},"settings":{"index":{"creation_date":"1531007073465","number_of_shards":"5","number_of_replicas":"1","uuid":"TBJYwFCMRb-iM9xkqdMp-g","version":{"created":"6030099"},"provided_name":"bupstest9"}}}}


#10

Finally, I have it working.
The problem was in my (mis)use of the "operator" construct within the queries, within the matches, within the must...
Thanks for your assistance.


(system) closed #11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.