# Invalid Results with whitespace analyzer and Multi match search

**URL:** <https://discuss.elastic.co/t/invalid-results-with-whitespace-analyzer-and-multi-match-search/172015>\
**Category:** Elasticsearch\
**Created:** [March 12, 2019, 5:35pm UTC](https://discuss.elastic.co/t/invalid-results-with-whitespace-analyzer-and-multi-match-search/172015 "2019-03-12T17:35:15Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![TARUN\_KUNDHIYA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarun_kundhiya/32/41839_2.png) [@TARUN\_KUNDHIYA](https://discuss.elastic.co/u/TARUN_KUNDHIYA)\
**Post date:** [March 12, 2019, 5:35pm UTC](https://discuss.elastic.co/t/invalid-results-with-whitespace-analyzer-and-multi-match-search/172015/1 "2019-03-12T17:35:15Z")

</div>

**These are only 3 documents in my index**

\</\>

```
"hits": [
{
"_index": "test_index",
"_type": "relationship",
"_id": "AWly5Q6mfOHUG_jfP5e8",
"_score": 1,
"_source": {
"fromEntityId": "a323dd-d43de-d43d3d4-d4334",
"toEntityId": "12345",
"score": 1
}
},
{
"_index": "test_index",
"_type": "relationship",
"_id": "AWly5Q6mfOHUG_jfP5e9",
"_score": 1,
"_source": {
"fromEntityId": "a323dd-48534-d43d3d4-hd4738f",
"toEntityId": "123455",
"score": 2
}
},
{
"_index": "test_index",
"_type": "relationship",
"_id": "AWly5Q6nfOHUG_jfP5e-",
"_score": 1,
"_source": {
"fromEntityId": "784hd4-48534-784d43-hd4738f",
"toEntityId": "1234556",
"score": 3
}
}
]

```

**When i analyze this text [2 whole keys space separated]**

```
{
  "text" : "784hd4-48534-784d43-hd4738f a323dd-48534-d43d3d4-hd4738f",
  "analyzer" : "whitespace"
}

I get these tokens which is okay

 {
            "token": "784hd4-48534-784d43-hd4738f",
            "start_offset": 0,
            "end_offset": 27,
            "type": "word",
            "position": 0
        },
        {
            "token": "a323dd-48534-d43d3d4-hd4738f",
            "start_offset": 28,
            "end_offset": 56,
            "type": "word",
            "position": 1
        }

```

**But in multi match search result i get nothing**

\</\>

```
{
  "query" : {
    "multi_match" : {
      "query" : "784hd4-48534-784d43-hd4738f 323dd-48534-d43d3d4-hd4738f",
	  "analyzer" : "whitespace",
      "fields" : ["fromEntityId" , "toEntityId"],
      "operator" : "or"
      }
  }
}

"hits": []

```

**This should return both those documents. Please help**

---

<div class="post-metadata">

**Author:** ![gbrown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gbrown/32/34482_2.png) [@gbrown](https://discuss.elastic.co/u/gbrown)\
**Post date:** [March 12, 2019, 11:42pm UTC](https://discuss.elastic.co/t/invalid-results-with-whitespace-analyzer-and-multi-match-search/172015/2 "2019-03-12T23:42:59Z")

</div>

What mapping is in use on the index? You generally shouldn't specify the analyzer at query time, it should be set on the field definition in the mapping - if the analyzer isn't set in the mapping, text fields will be indexed using the `standard` analyzer, which will split based on the `-` characters as well. I suspect that's what's happening in this case - can you try re-creating your index with the `analyzer` field set to `whitespace` in the mapping, similar to the example [shown here](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/analyzer.html) and re-try the query?

---

<div class="post-metadata">

**Author:** ![TARUN\_KUNDHIYA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarun_kundhiya/32/41839_2.png) [@TARUN\_KUNDHIYA](https://discuss.elastic.co/u/TARUN_KUNDHIYA)\
**Post date:** [March 17, 2019, 9:54am UTC](https://discuss.elastic.co/t/invalid-results-with-whitespace-analyzer-and-multi-match-search/172015/3 "2019-03-17T09:54:53Z")

</div>

Hi Gordon, thanks.

We tried PUTTING custom analyzer and then re indexed all our indices. Now it works without specifying the analyzer while querying.

But is there any support that we can specify the analyzer during the query time?

This doc summarizes something like that.  
[https://www.elastic.co/guide/en/elasticsearch/reference/current/search-analyzer.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-analyzer.html)

Thus we believe there should be something to override the index analyzers and give us the flexibility to use different analyzer chosen at runtime.

And it is very hard to re create indices at large scale. Please let us know if there exists any solution/hack for this.

---

<div class="post-metadata">

**Author:** ![gbrown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gbrown/32/34482_2.png) [@gbrown](https://discuss.elastic.co/u/gbrown)\
**Post date:** [March 18, 2019, 8:34pm UTC](https://discuss.elastic.co/t/invalid-results-with-whitespace-analyzer-and-multi-match-search/172015/4 "2019-03-18T20:34:21Z")

</div>

I'm glad that helped out! I think to help clarify why what you were doing before wasn't working, we're going to have to dive into how Elasticsearch indexes text.

Specifying an analyzer at query time is supported, but what's critical to note is that if you specify an analyzer at query time, that analyzer will only be applied to the query, not to the data already in the index.

Here's what I mean by that: When you index a document in Elasticsearch, the `text` fields in that document are analyzed using either the specified analyzer or the default `standard` analyzer if no analyzer is specified. The output of the analyzer is a list of tokens, and those tokens are what is used to look up the document.

To use your example, when you indexed the documents into Elasticsearch, the `standard` analyzer was used, and got tokenized like this:

```auto
{
  "tokens" : [
    {
      "token" : "784hd4",
      "start_offset" : 0,
      "end_offset" : 6,
      "type" : "<ALPHANUM>",
      "position" : 0
    },
    {
      "token" : "48534",
      "start_offset" : 7,
      "end_offset" : 12,
      "type" : "<NUM>",
      "position" : 1
    },
    {
      "token" : "784d43",
      "start_offset" : 13,
      "end_offset" : 19,
      "type" : "<ALPHANUM>",
      "position" : 2
    },
    {
      "token" : "hd4738f",
      "start_offset" : 20,
      "end_offset" : 27,
      "type" : "<ALPHANUM>",
      "position" : 3
    }
  ]
}

```

And those tokens were used to build the inverted index. So you could search for `78d43`, and that document would get returned.

But when you use the `whitespace` analyzer in the query, the search was analyzed like this:

```auto
{
  "tokens" : [
    {
      "token" : "784hd4-48534-784d43-hd4738f",
      "start_offset" : 0,
      "end_offset" : 27,
      "type" : "word",
      "position" : 0
    }
  ]
}

```

As one token. And that one token, `784hd4-48534-784d43-hd4738f` wasn't in the index (which would have contained the smaller tokens [`784hd4`, `48534`, `784d43`, `hd4738f`] the string was broken up into by the `standard` analyzer), so no documents were returned. This is why the same analyzer is usually used at index and at search time, so that the same string generates the same set of tokens.

The analyzer in the mapping controls how data is indexed, and changing that after the fact is very expensive - it would effectively just be a reindex, as Elasticsearch would have to re-analyze every document and rebuild the index every time the analyzer is changed.

If you want to know more, I highly recommend the [Inverted Index](https://www.elastic.co/guide/en/elasticsearch/guide/2.x/inverted-index.html) and [Analysis and Analyzers](https://www.elastic.co/guide/en/elasticsearch/guide/2.x/analysis-intro.html) sections of the Elasticsearch Definitive Guide - the guide itself is a bit out of date, but those sections are still highly relevant to how Elasticsearch stores and searches data.

---

<div class="post-metadata">

**Author:** ![gbrown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gbrown/32/34482_2.png) [@gbrown](https://discuss.elastic.co/u/gbrown)\
**Post date:** [March 18, 2019, 8:36pm UTC](https://discuss.elastic.co/t/invalid-results-with-whitespace-analyzer-and-multi-match-search/172015/5 "2019-03-18T20:36:33Z")

</div>

As a follow-up to this, if you want to index the same text multiple ways (e.g. using different analyzers), one way to do this is using [multi-fields](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/multi-fields.html). Depending on your use case, that may be helpful - you'll still need to set up the analyzers you need ahead of time, but you can switch between them by choosing which sub-field to query.

---

<div class="post-metadata">

**Author:** ![TARUN\_KUNDHIYA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarun_kundhiya/32/41839_2.png) [@TARUN\_KUNDHIYA](https://discuss.elastic.co/u/TARUN_KUNDHIYA)\
**Post date:** [March 20, 2019, 2:23pm UTC](https://discuss.elastic.co/t/invalid-results-with-whitespace-analyzer-and-multi-match-search/172015/6 "2019-03-20T14:23:42Z")

</div>

Thank you Gordon for the detailed explanation and the multi-fields usage tip.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 17, 2019, 2:23pm UTC](https://discuss.elastic.co/t/invalid-results-with-whitespace-analyzer-and-multi-match-search/172015/7 "2019-04-17T14:23:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.