# Ngram search always returns same document

**URL:** <https://discuss.elastic.co/t/ngram-search-always-returns-same-document/54426>\
**Category:** Elasticsearch\
**Created:** [June 30, 2016, 3:24pm UTC](https://discuss.elastic.co/t/ngram-search-always-returns-same-document/54426 "2016-06-30T15:24:40Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mawalu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mawalu/32/10653_2.png) [@Mawalu](https://discuss.elastic.co/u/Mawalu)\
**Post date:** [June 30, 2016, 3:24pm UTC](https://discuss.elastic.co/t/ngram-search-always-returns-same-document/54426/1 "2016-06-30T15:24:40Z")

</div>

I currently build a search for a contact list. The user should be allowed to search on any property and get results without typing full words / tokens. After I researched on the topic I found the ngram filter and tried to configure it as shown below:

```
 $params = [
        'index' => SearchPerson::getIndex(),
        'body' => [
            'settings' => [
                "analysis" => [
                    "analyzer" => [
                        "ngram_analyzer" => [
                            "type" => "custom",
                            "tokenizer" => "whitespace",
                            "filter" => ["asciifolding", "lowercase", "ngram"]
                        ],
                        "whitespace_analyzer" => [
                            "type" => "custom",
                            "tokenizer" => "whitespace",
                            "filter" => [
                                "lowercase",
                                "asciifolding"
                            ]
                        ]
                    ],
                    "filter" => [
                        "ngram" => [
                            "type" => "ngram",
                            "min_gram" => 2,
                            "max_gram" => 20,
                            "token_chars" => [
                                "letter",
                                "digit",
                                "punctuation",
                                "symbol"
                            ]
                        ]
                    ]
                ]
            ],
            'mapping' => [
                'person' => [
                    '_all' => [
                        'index_analyzer' => 'ngram_analyzer',
                        'search_analyzer' => 'whitespace_analyzer'
                    ],
                    'properties' => [
                        'firstname' => [
                            'index_analyzer' => 'ngram_analyzer',
                            'search_analyzer' => 'whitespace_analyzer'
                        ]
                    ]
                ]
            ]
        ]
    ];

```

When I query the index using this query:

```
{
  "size": 10,
  "query": {
    "match": {
      "_all": {
        "query": "Terill",
      }
    }
  }
}

```

i always get this document in return and not the one I intended:

```
{
[...]
        "firstname": "Austen",
        "lastname": "Braun",
[...]
}

```

([full document](http://pastebin.com/dyTsGyvR))

The one I intend to find is the following one:

```
{
[...]
        "firstname": "Terrill",
        "lastname": "Parker",
[...]
}

```

([full document](http://pastebin.com/egTBd7Cr))

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [July 4, 2016, 8:18am UTC](https://discuss.elastic.co/t/ngram-search-always-returns-same-document/54426/2 "2016-07-04T08:18:21Z")

</div>

Hey,

the \_all field contains values from all of your fields. So if Austen Braun lives in the city of `Terill`, then this document will match, unless you specify to only query for the name fields. You need to provide the full document here to debug this further...

--Alex

---

<div class="post-metadata">

**Author:** ![Mawalu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mawalu/32/10653_2.png) [@Mawalu](https://discuss.elastic.co/u/Mawalu)\
**Post date:** [July 4, 2016, 1:21pm UTC](https://discuss.elastic.co/t/ngram-search-always-returns-same-document/54426/3 "2016-07-04T13:21:39Z")

</div>

I know that this could be a problem and that the full documents would help. Since I hit the character limit with my post I uploaded the documents to pastebin and posted the link in my post.

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [July 5, 2016, 6:51am UTC](https://discuss.elastic.co/t/ngram-search-always-returns-same-document/54426/4 "2016-07-05T06:51:48Z")

</div>

Hey,

I just tried this locally and it seemed to work as expected. Can you please create and provide an exact curl recreation (no other programming languages, like your mapping above, see [https://www.elastic.co/help](https://www.elastic.co/help)) from mapping/index creation, to indexing documents up to searching and specify the elasticsearch version you are trying this out on.

--Alex

---

<div class="post-metadata">

**Author:** ![Mawalu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mawalu/32/10653_2.png) [@Mawalu](https://discuss.elastic.co/u/Mawalu)\
**Post date:** [July 5, 2016, 5:13pm UTC](https://discuss.elastic.co/t/ngram-search-always-returns-same-document/54426/5 "2016-07-05T17:13:42Z")

</div>

Thanks for your response. I created a gist [here](https://gist.github.com/Mawalu/9dee10a97fcfdec133729737b27d034b) with every request and the curl commands I used.

I first used the head plugin which strangely produced some different results. For example a query for the term `Aust` returned a result when I created everything using head but not when using curl. When I check the mapping ngram and everything is configured as always.

Also I'm on Arch Linux and this is the output of --version:  
`Version: 2.3.3, Build: 218bdf1/2016-05-17T15:40:04Z, JVM: 1.8.0_92`

Greetings Martin

---

<div class="post-metadata">

**Author:** ![Phrobe](https://avatars.discourse-cdn.com/v4/letter/p/3ab097/32.png) [@Phrobe](https://discuss.elastic.co/u/Phrobe)\
**Post date:** [July 11, 2016, 5:05pm UTC](https://discuss.elastic.co/t/ngram-search-always-returns-same-document/54426/6 "2016-07-11T17:05:08Z")

</div>

@Mawalu Did you solve the problem already? I am trying to implement something similar.

---

<div class="post-metadata">

**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\
**Post date:** [July 11, 2016, 6:52pm UTC](https://discuss.elastic.co/t/ngram-search-always-returns-same-document/54426/7 "2016-07-11T18:52:36Z")

</div>

I have not fully looked into your example, but one thing that sticks out is  
the use of "index\_analyzer", which I believe has been fully removed in 2.x:  
[https://www.elastic.co/guide/en/elasticsearch/reference/2.3/breaking\_20\_mapping\_changes.html#\_analyzer\_mappings](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/breaking_20_mapping_changes.html#_analyzer_mappings)

Try changing the field analyzer setting and see if it works.

---

<div class="post-metadata">

**Author:** ![Mawalu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mawalu/32/10653_2.png) [@Mawalu](https://discuss.elastic.co/u/Mawalu)\
**Post date:** [July 12, 2016, 1:33pm UTC](https://discuss.elastic.co/t/ngram-search-always-returns-same-document/54426/8 "2016-07-12T13:33:00Z")

</div>

Thanks for your answer.

I changed my mappings but I still have the same results. I also updated [my gist](https://gist.github.com/Mawalu/9dee10a97fcfdec133729737b27d034b) to reflect the changes

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:36pm UTC](https://discuss.elastic.co/t/ngram-search-always-returns-same-document/54426/9 "2017-07-05T22:36:17Z")

</div>


