# Simple\_query\_string and multi terms synonyms

**URL:** <https://discuss.elastic.co/t/simple-query-string-and-multi-terms-synonyms/174780>\
**Category:** Elasticsearch\
**Created:** [April 1, 2019, 12:30pm UTC](https://discuss.elastic.co/t/simple-query-string-and-multi-terms-synonyms/174780 "2019-04-01T12:30:39Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Fabrizio\_Fortino](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fabrizio_fortino/32/1719_2.png) [@Fabrizio\_Fortino](https://discuss.elastic.co/u/Fabrizio_Fortino)\
**Post date:** [April 1, 2019, 12:30pm UTC](https://discuss.elastic.co/t/simple-query-string-and-multi-terms-synonyms/174780/1 "2019-04-01T12:30:39Z")

</div>

I have a field with the following search\_analyzer:

```
"name_search_en" : {
   "filter" : [
     "english_possessive_stemmer",
     "lowercase",
     "name_synonyms_en",
     "english_stop",
     "english_stemmer",
     "asciifolding"
   ],
   "tokenizer" : "standard"
}

```

`name_synonyms_en` is a `synonym_graph` that looks like this

```
"name_synonyms_en" : {
  "type" : "synonym_graph",
   "synonyms" : [
      "beach bag => straw bag,beach bag",
      "bicycle,bike"
    ]
 }

```

Running the following `multi_match` query the synonym are correctly applied

```
{
  "query": {
    "multi_match": {
      "query": "beach bag",
      "auto_generate_synonyms_phrase_query": false,
      "type": "cross_fields",
      "fields": [
        "brand.en-US^1.0",
        "name.en-US^1.0"
      ]
    }
  }
}

```

Here is the `_validate` explanation output. Both `beach bag` and `straw bag` are present, as expected, in the raw query:

```
"explanations" : [
{
  "index" : "d7598351-311f-4844-bb91-4f26c9f538f3",
  "valid" : true,
  "explanation" : "+((((+name.en-US:straw +name.en-US:bag) (+name.en-US:beach +name.en-US:bag))) | (brand.en-US:beach brand.en-US:bag)) #DocValuesFieldExistsQuery [field=_primary_term]"
}

```

]

I would expect the same in the following `simple_query_string`

```
{
  "query": {
    "simple_query_string": {
      "query": "beach bag",
      "auto_generate_synonyms_phrase_query": false,
      "fields": [
        "brand.en-US^1.0",
        "name.en-US^1.0"
      ]
    }
  }
}

```

but the `straw bag` synonym is not present in the raw query

```
"explanations" : [
{
  "index" : "d7598351-311f-4844-bb91-4f26c9f538f3",
  "valid" : true,
  "explanation" : "+((name.en-US:beach | brand.en-US:beach)~1.0 (name.en-US:bag | brand.en-US:bag)~1.0) #DocValuesFieldExistsQuery [field=_primary_term]"
}
]

```

The problem seems to be related to multi-terms synonyms only. If I search for `bike`, the `bicycle` synonym is correctly present in the query

```
"explanations" : [
{
  "index" : "d7598351-311f-4844-bb91-4f26c9f538f3",
  "valid" : true,
  "explanation" : "+(Synonym(name.en-US:bicycl name.en-US:bike) | brand.en-US:bike)~1.0 #DocValuesFieldExistsQuery [field=_primary_term]"
}
]

```

Is this the expected behaviour (meaning multi terms synonyms are not supported for this query)?

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [April 4, 2019, 6:54am UTC](https://discuss.elastic.co/t/simple-query-string-and-multi-terms-synonyms/174780/2 "2019-04-04T06:54:00Z")

</div>

The simple\_query\_string query parses the query string by breaking on (amongst other things) whitespace, before that string is analyzed. As a result, the analyzer sees the individual tokens "beach" and "bag", but not the synonym "beach bag".

You can change this behavior with the [`flags` parameter](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html#supported-flags). This defaults to `ALL`, but you could change that into a subset that does not include `WHITESPACE`. For example:

```auto
{
  "query": {
    "simple_query_string": {
      "query": "beach bag",
      "auto_generate_synonyms_phrase_query": false,
      "flags": "OR|AND|PREFIX", 
      "fields": [
        "brand.en-US^1.0",
        "name.en-US^1.0"
      ]
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Fabrizio\_Fortino](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fabrizio_fortino/32/1719_2.png) [@Fabrizio\_Fortino](https://discuss.elastic.co/u/Fabrizio_Fortino)\
**Post date:** [April 4, 2019, 10:10am UTC](https://discuss.elastic.co/t/simple-query-string-and-multi-terms-synonyms/174780/3 "2019-04-04T10:10:05Z")

</div>

Thank you!

This solves the multiwords synonym problem but introduces another issue. If I don't specify `flags` the match query works on the specified fields (`brand.en-US` & `name.en-US`) as expected. When I specify `flags` (I tried with `NONE` and `OR|AND|PREFIX` as values) the match seems to work on all the documents' fields while the `fields` are used only for relevance (like the `should` clause in the bool query). Any idea. Thanks!

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [April 4, 2019, 11:37am UTC](https://discuss.elastic.co/t/simple-query-string-and-multi-terms-synonyms/174780/4 "2019-04-04T11:37:45Z")

</div>

I'm sorry, but I don't understand the issue. What is it that you're trying to achieve?

---

<div class="post-metadata">

**Author:** ![Fabrizio\_Fortino](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fabrizio_fortino/32/1719_2.png) [@Fabrizio\_Fortino](https://discuss.elastic.co/u/Fabrizio_Fortino)\
**Post date:** [April 4, 2019, 1:49pm UTC](https://discuss.elastic.co/t/simple-query-string-and-multi-terms-synonyms/174780/5 "2019-04-04T13:49:47Z")

</div>

Please don't take into account my previous reply. The issue is somewhere else.  
Let me try to summarise what I want to do: I am trying to do a full text search on two fields with different synonym\_graph configurations (`productTypes.en-US` and `name.en-US`).

Here is the full query:

```
{
  "query": {
    "simple_query_string": {
      "query": "beach bag",
      "auto_generate_synonyms_phrase_query": false,
      "minimum_should_match": "2<66%",
      "fields": [
        "name.en-US^1.0",
       "productTypes.en-US^1.0"
      ]
    }
  }
}

```

I dig into the raw queries a bit and I think the issue is related to the `minimum_should_match`. The above query resolves to this

> +(((productTypes.en-US:beach | name.en-US:beach)~1.0 (productTypes.en-US:bag | name.en-US:bag)~1.0)~2) #DocValuesFieldExistsQuery [field=\_primary\_term]

From my understanding the `~2` at the end of the raw query is the `minimum_should_match` value. When I specify the `"flags": "OR|AND|PREFIX"` as you suggested the query is resolved to

> +((productTypes.en-US:beach productTypes.en-US:bag) | (((+name.en-US:straw +name.en-US:bag) (+name.en-US:beach +name.en-US:bag))))~1.0 #DocValuesFieldExistsQuery [field=\_primary\_term]

As you can see, the multiwords synonyms are correctly applied but the `minimum_should_match` is not included anymore.

Is there something wrong with my query or is this an issue with `simple_query_string` ?

Thanks!

---

<div class="post-metadata">

**Author:** ![Fabrizio\_Fortino](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fabrizio_fortino/32/1719_2.png) [@Fabrizio\_Fortino](https://discuss.elastic.co/u/Fabrizio_Fortino)\
**Post date:** [April 5, 2019, 10:44am UTC](https://discuss.elastic.co/t/simple-query-string-and-multi-terms-synonyms/174780/6 "2019-04-05T10:44:26Z")

</div>

@abdon

I have created this test case to better explain the problem

> **[elastic/elasticsearch](https://github.com/elastic/elasticsearch/compare/master...fabriziofortino:simple-query-string-msm-issue)**
>
> Open Source, Distributed, RESTful Search Engine. Contribute to elastic/elasticsearch development by creating an account on GitHub.

The first assert (line 305) fails because the number of hits is 3 since the minimum\_should\_match is not applied.

Should I file an issue in the elasticsearch repo?

Thanks

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [April 5, 2019, 11:53am UTC](https://discuss.elastic.co/t/simple-query-string-and-multi-terms-synonyms/174780/7 "2019-04-05T11:53:16Z")

</div>

I don't think this is a bug, but this is going deep into the internals, so someone else may be better suited to explain this.

My understanding is that if you're not splitting the query string on whitespace, then you're sending Elasticsearch one query clause. With one query clause, minimum\_should\_match does not make sense. (The [docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html) say this about that: "`minimum_should_match`: The minimum number of clauses that must match for a document to be returned. See the minimum\_should\_match documentation for the full list of options.")

Why use the simple\_query\_string query at all? Are you giving your users the capability to query with `AND` and `OR` operators? If not, the match and multi\_match queries may be easier to use, as the query parsing of those queries will be simpler. If you do need the simple\_query\_string query, maybe you can create a bool query with one simple\_query\_string query that splits on white space, and one that does not? That way you can make use of both functionalities.

---

<div class="post-metadata">

**Author:** ![Fabrizio\_Fortino](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fabrizio_fortino/32/1719_2.png) [@Fabrizio\_Fortino](https://discuss.elastic.co/u/Fabrizio_Fortino)\
**Post date:** [April 5, 2019, 2:20pm UTC](https://discuss.elastic.co/t/simple-query-string-and-multi-terms-synonyms/174780/8 "2019-04-05T14:20:37Z")

</div>

I have tried to use a `multi_match` query but it does not work in my case because `minimum_should_match` is applied to each field individually (see [here](https://www.elastic.co/guide/en/elasticsearch/reference/6.7/query-dsl-multi-match-query.html#operator-min)). `cross_field` would not work either because, as the [docs](https://www.elastic.co/guide/en/elasticsearch/reference/6.7/query-dsl-multi-match-query.html#_literal_cross_field_literal_and_analysis) say "can only work in term-centric mode on fields that have the same analyzer". Having different analyzers (with different `synonym_graph` configurations), would make the query work like a `most_fields` query.

A workaround for this would be to setup synonyms at index time but I would prefer to not go down that road.

I thought using `query_string` or `simple_query_string` I could find a suitable solution for that. I dig a bit more into the code and I debug the the `SimpleQueryParser`.

This is what I found: with `WHITESPACE` flag the parser creates a boolean query with a clause for each token. As an example, the `simple_qeury_string` in the unit test I have created generates this bool query:

```
((otherbody:foo | body:bar)~1.0 (otherbody:bar | body:bar)~1.0)~2)

```

Since boolean queries support `minimum_should_match` the parameter is correctly applied.

The same query WITHOUT `WHITESPACE` generates the following `DisjunctionMaxQuery`

```
((otherbody:foo otherbody:bar) | (body:foo body:bar))~1.0

```

As you can see, the original text (`foo bar`) gets split but the `minimum_should_match` is not applied because `dismax` does not support it.

If `dismax` could be extended to support `minimum_should_match` it would solve this issue.

---

<div class="post-metadata">

**Author:** ![Fabrizio\_Fortino](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fabrizio_fortino/32/1719_2.png) [@Fabrizio\_Fortino](https://discuss.elastic.co/u/Fabrizio_Fortino)\
**Post date:** [April 5, 2019, 4:45pm UTC](https://discuss.elastic.co/t/simple-query-string-and-multi-terms-synonyms/174780/9 "2019-04-05T16:45:13Z")

</div>

Ok. I think this is the expected behaviour. This logic is well explained in this [docs paragraph](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#_multi_field_2).

> in multi-fields query the `minimum_should_match` parameter can’t be applied.

This is a pity since this makes impossible to use multiple fields with synonyms and the minimum\_should\_match parameter.

We are migrating from solr where [dismax](https://lucene.apache.org/solr/guide/7_4/the-dismax-query-parser.html) has support Minimum Match. It would be nice to have something like that in elasticsearch or at least some alternative way to get the same results. Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 3, 2019, 4:45pm UTC](https://discuss.elastic.co/t/simple-query-string-and-multi-terms-synonyms/174780/10 "2019-05-03T16:45:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.