# Suggestions for person names doens't work so well

**URL:** <https://discuss.elastic.co/t/suggestions-for-person-names-doenst-work-so-well/248035>\
**Category:** Elasticsearch\
**Created:** [September 9, 2020, 3:38pm UTC](https://discuss.elastic.co/t/suggestions-for-person-names-doenst-work-so-well/248035 "2020-09-09T15:38:06Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Daniele\_Renda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniele_renda/32/39939_2.png) [@Daniele\_Renda](https://discuss.elastic.co/u/Daniele_Renda)\
**Post date:** [September 9, 2020, 3:38pm UTC](https://discuss.elastic.co/t/suggestions-for-person-names-doenst-work-so-well/248035/1 "2020-09-09T15:38:06Z")

</div>

Hi guys,  
I know that this use case is quite common but I'm a bit confused after I've read several tutorials and docs.  
In my application I'm trying to help the user when he digits a person's name in the form. I'd like to show a suggestion/autocomplete to:

1. make insert faster
2. reduce mispelled names

I've an index with all common names in my country.  
In the first instance I used a fuzzy search but I soon realized this is not what I want because during typing results are not what I expect.  
For example when I write "Dan", I'd like to see also "Daniel" as result. I need an "autocomplete with fuzzy search". I don't really know if is so simple to show smart autocomplete for names because there are several things to consider: I wrote partial name, I wrote a mispelled name.

I saw [suggesters](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters.html), [fuzzy query](https://abhishek376.wordpress.com/2017/09/30/correcting-typos-and-spelling-mistakes-using-elasticsearch/)....

If you could point me in the right direction I'd appreciate.

Thanks

---

<div class="post-metadata">

**Author:** ![Daniele\_Renda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniele_renda/32/39939_2.png) [@Daniele\_Renda](https://discuss.elastic.co/u/Daniele_Renda)\
**Post date:** [September 10, 2020, 7:39am UTC](https://discuss.elastic.co/t/suggestions-for-person-names-doenst-work-so-well/248035/2 "2020-09-10T07:39:11Z")

</div>

After a bit of tries I came up with something even if I'm not completely satisfied of results.

This is my mapping:

```
{
  "properties": {
    "name": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword"
        }
      }
    },
    "name_suggest": {
      "type": "completion",
      "contexts": [
        {
          "name": "country_context",
          "type": "category",
          "path": "country.keyword"
        }
      ]
    },
    "gender": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword"
        }
      }
    },
    "country": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword"
        }
      }
    }
  }
}

```

An example of data is this:  
 ![Senzanome](https://us1.discourse-cdn.com/elastic/original/3X/0/9/09778f18849962ead346fd7e0519faf02c248674.png)

I want to show suggestions to the user during typing using also fuzzy to allow some typo.  
I came up with this query that uses suggestions AND exact match because I want to give a better score to exact match rather than suggestions.

```
{
  "size": 15,
  "query": {
    "bool": {
      "should": [
        {
          "match": {
            "name.keyword": {
              "query": "Marc",
              "operator": "OR",
              "prefix_length": 0,
              "max_expansions": 50,
              "fuzzy_transpositions": true,
              "lenient": false,
              "zero_terms_query": "NONE",
              "auto_generate_synonyms_phrase_query": true,
              "boost": 1
            }
          }
        }
      ],
      "adjust_pure_negative": true,
      "boost": 1
    }
  },
  "_source": {
    "includes": [
      "name"
    ],
    "excludes": []
  },
  "suggest": {
    "text": "Marc",
    "complete": {
      "text": "Marc",
      "prefix": "Marc",
      "completion": {
        "field": "name_suggest",
        "size": 10,
        "fuzzy": {
          "fuzziness": 1,
          "transpositions": true,
          "min_length": 3,
          "prefix_length": 1,
          "unicode_aware": false,
          "max_determinized_states": 10000
        },
        "contexts": {
          "country_context": [
            {
              "context": "IT",
              "boost": 1,
              "prefix": false
            }
          ]
        }
      }
    }
  }
}

```

and these are results:

```
{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 0,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  },
  "suggest": {
    "complete": [
      {
        "text": "Marc",
        "offset": 0,
        "length": 4,
        "options": [
          {
            "text": "Mara",
            "_index": "personname",
            "_type": "_doc",
            "_id": "2046830049",
            "_score": 4,
            "_routing": "global",
            "_source": {
              "name": "Mara"
            },
            "contexts": {
              "country_context": [
                "IT"
              ]
            }
          },
          {
            "text": "Marat",
            "_index": "personname",
            "_type": "_doc",
            "_id": "-1718195506",
            "_score": 4,
            "_routing": "global",
            "_source": {
              "name": "Marat"
            },
            "contexts": {
              "country_context": [
                "IT"
              ]
            }
          },
          {
            "text": "Marca",
            "_index": "personname",
            "_type": "_doc",
            "_id": "-2041994534",
            "_score": 4,
            "_routing": "global",
            "_source": {
              "name": "Marca"
            },
            "contexts": {
              "country_context": [
                "IT"
              ]
            }
          },
          {
            "text": "Marcantonio",
            "_index": "personname",
            "_type": "_doc",
            "_id": "-16856444",
            "_score": 4,
            "_routing": "global",
            "_source": {
              "name": "Marcantonio"
            },
            "contexts": {
              "country_context": [
                "IT"
              ]
            }
          },
          {
            "text": "Marcella",
            "_index": "personname",
            "_type": "_doc",
            "_id": "48281663",
            "_score": 4,
            "_routing": "global",
            "_source": {
              "name": "Marcella"
            },
            "contexts": {
              "country_context": [
                "IT"
              ]
            }
          },
          {
            "text": "Marcelliano",
            "_index": "personname",
            "_type": "_doc",
            "_id": "-836086954",
            "_score": 4,
            "_routing": "global",
            "_source": {
              "name": "Marcelliano"
            },
            "contexts": {
              "country_context": [
                "IT"
              ]
            }
          },
          {
            "text": "Marcellina",
            "_index": "personname",
            "_type": "_doc",
            "_id": "-695286534",
            "_score": 4,
            "_routing": "global",
            "_source": {
              "name": "Marcellina"
            },
            "contexts": {
              "country_context": [
                "IT"
              ]
            }
          },
          {
            "text": "Marcellino",
            "_index": "personname",
            "_type": "_doc",
            "_id": "-371432729",
            "_score": 4,
            "_routing": "global",
            "_source": {
              "name": "Marcellino"
            },
            "contexts": {
              "country_context": [
                "IT"
              ]
            }
          },
          {
            "text": "Marcello",
            "_index": "personname",
            "_type": "_doc",
            "_id": "372135468",
            "_score": 4,
            "_routing": "global",
            "_source": {
              "name": "Marcello"
            },
            "contexts": {
              "country_context": [
                "IT"
              ]
            }
          },
          {
            "text": "Marchetto",
            "_index": "personname",
            "_type": "_doc",
            "_id": "-1073596950",
            "_score": 4,
            "_routing": "global",
            "_source": {
              "name": "Marchetto"
            },
            "contexts": {
              "country_context": [
                "IT"
              ]
            }
          }
        ]
      }
    ]
  }
}

```

**What I expected:**

I expected to have in results at least "Marco" (italian name).

**What I got:**

I got these suggestions from ES that are quite far from what the user wants:

```
[
    "Mara",
    "Marat",
    "Marca",
    "Marcantonio",
    "Marcella",
    "Marcelliano",
    "Marcellina",
    "Marcellino",
    "Marcello",
    "Marchetto"
]

```

I don't get why "Marco" that is closer to the search string "Marc" is not selected.  
**A small detail:** If I increase the size of results from 10 to 20 I got this response that contains "Marco":

```
[
    "Mara",
    "Marat",
    "Marca",
    "Marcantonio",
    "Marcella",
    "Marcelliano",
    "Marcellina",
    "Marcellino",
    "Marcello",
    "Marchetto,
    "Marchina",
    "Marchino",
    "Marchisio",
    "Marciano",
    "Marciliano",
    "Marcilio",
    "Marco",
    "Marcolina",
    "Marcolino",
    "Marcuccia"
]

```

but the sorting of results are not good enough and I don't understand how I can improve that results.

I hope in some hint. Thanks

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [September 10, 2020, 8:58am UTC](https://discuss.elastic.co/t/suggestions-for-person-names-doenst-work-so-well/248035/3 "2020-09-10T08:58:36Z")

</div>

I suspect it will be hard to have a single request that both completes partial names and spell-checks sensibly. A spell check usually starts with the assumption that words are complete.

> [@Daniele\_Renda](#):
>
> reduce mispelled names

You can match names on the basis of

1. How they look
2. How they sound
3. How they're actually used.

The problem with 1) is that simple fuzzy matching (edit distance) is not perfect. It's a short distance from `john` to `joan` and a long distance from `bob` to `robert`

The problem with 2) is that while we have phonetic analyzers they use exact-code matching only (there's no degrees of fuzziness) and they can have many false positives and false negatives.

Option 3 is the best but relies on a lot of data. If you have a load of strong IDs e.g customer account numbers/email addresses etc and for each, a bunch of associated names (including noise like typos, misheard pronunciations, shortened names etc) then you can machine-learn a thesaurus like this:

 ![names4.tgf - yEd](https://us1.discourse-cdn.com/elastic/original/3X/5/8/5823f750ee6bd9acfed3e9e1fb0486f57c9826c3.jpeg)

This is a weighted graph that can tell you `janes` is statistically more strongly associated with `james` than `jane` (probably because the `n` is next to `m` on the keyboard).  
Not everyone is lucky enough to have these piles of data though which is why firms like Basis Technology offer specialised solutions.

It's not an easy problem.

---

<div class="post-metadata">

**Author:** ![Daniele\_Renda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniele_renda/32/39939_2.png) [@Daniele\_Renda](https://discuss.elastic.co/u/Daniele_Renda)\
**Post date:** [September 10, 2020, 1:54pm UTC](https://discuss.elastic.co/t/suggestions-for-person-names-doenst-work-so-well/248035/4 "2020-09-10T13:54:14Z")

</div>

Thanks for your really interesting explanation. I see that is really complex the last solution even if it would be the best of course.

Unfortunately I don't have such data and I should accept a compromise. I suppose a simpler approach would be to:

1. Use search as you type when the user is writing the name, trying to anticipate what he is writing
2. Run a Suggester like what I'm doing now, **after** the user wrote the whole word, and check if what the user wrote is included in the ES' suggestion list. If it's included it means the name is right, otherwise I can show some UI component saying "Did you mean \<\>?"

What do you think about? Thanks very much

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [September 10, 2020, 4:08pm UTC](https://discuss.elastic.co/t/suggestions-for-person-names-doenst-work-so-well/248035/5 "2020-09-10T16:08:18Z")

</div>

That makes perfect sense to me.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 8, 2020, 4:08pm UTC](https://discuss.elastic.co/t/suggestions-for-person-names-doenst-work-so-well/248035/6 "2020-10-08T16:08:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.