Ngram search always returns same document

Elastic Stack Elasticsearch
1

I currently build a search for a contact list. The user should be allowed to search on any property and get results without typing full words / tokens. After I researched on the topic I found the ngram filter and tried to configure it as shown below:

 $params = [
        'index' => SearchPerson::getIndex(),
        'body' => [
            'settings' => [
                "analysis" => [
                    "analyzer" => [
                        "ngram_analyzer" => [
                            "type" => "custom",
                            "tokenizer" => "whitespace",
                            "filter" => ["asciifolding", "lowercase", "ngram"]
                        ],
                        "whitespace_analyzer" => [
                            "type" => "custom",
                            "tokenizer" => "whitespace",
                            "filter" => [
                                "lowercase",
                                "asciifolding"
                            ]
                        ]
                    ],
                    "filter" => [
                        "ngram" => [
                            "type" => "ngram",
                            "min_gram" => 2,
                            "max_gram" => 20,
                            "token_chars" => [
                                "letter",
                                "digit",
                                "punctuation",
                                "symbol"
                            ]
                        ]
                    ]
                ]
            ],
            'mapping' => [
                'person' => [
                    '_all' => [
                        'index_analyzer' => 'ngram_analyzer',
                        'search_analyzer' => 'whitespace_analyzer'
                    ],
                    'properties' => [
                        'firstname' => [
                            'index_analyzer' => 'ngram_analyzer',
                            'search_analyzer' => 'whitespace_analyzer'
                        ]
                    ]
                ]
            ]
        ]
    ];

When I query the index using this query:

{
  "size": 10,
  "query": {
    "match": {
      "_all": {
        "query": "Terill",
      }
    }
  }
}

i always get this document in return and not the one I intended:

{
[...]
        "firstname": "Austen",
        "lastname": "Braun",
[...]
}

(full document)

The one I intend to find is the following one:

{
[...]
        "firstname": "Terrill",
        "lastname": "Parker",
[...]
}

(full document)

2

Hey,

the _all field contains values from all of your fields. So if Austen Braun lives in the city of Terill, then this document will match, unless you specify to only query for the name fields. You need to provide the full document here to debug this further...

--Alex

3

I know that this could be a problem and that the full documents would help. Since I hit the character limit with my post I uploaded the documents to pastebin and posted the link in my post.

4

Hey,

I just tried this locally and it seemed to work as expected. Can you please create and provide an exact curl recreation (no other programming languages, like your mapping above, see https://www.elastic.co/help) from mapping/index creation, to indexing documents up to searching and specify the elasticsearch version you are trying this out on.

--Alex

5

Thanks for your response. I created a gist here with every request and the curl commands I used.

I first used the head plugin which strangely produced some different results. For example a query for the term Aust returned a result when I created everything using head but not when using curl. When I check the mapping ngram and everything is configured as always.

Also I'm on Arch Linux and this is the output of --version:
Version: 2.3.3, Build: 218bdf1/2016-05-17T15:40:04Z, JVM: 1.8.0_92

Greetings Martin

6

@Mawalu Did you solve the problem already? I am trying to implement something similar.

7

I have not fully looked into your example, but one thing that sticks out is
the use of "index_analyzer", which I believe has been fully removed in 2.x:
https://www.elastic.co/guide/en/elasticsearch/reference/2.3/breaking_20_mapping_changes.html#_analyzer_mappings

Try changing the field analyzer setting and see if it works.

8

Thanks for your answer.

I changed my mappings but I still have the same results. I also updated my gist to reflect the changes