Dec 6th, 2022: [EN] A completion suggester for Santa's little helpers

[Alexis_Roberson]

A completion suggester for Santa’s little helpers

• Introduction
• What is a completion suggester?
• Major components of a completion suggester
• Walkthrough / Demo
• Drawbacks
• Conclusion

Introduction

The elves have spun up their elastic cluster to search for the toys to be made for each child. With Christmas around the corner and the need to get this done quickly, they would like to implement a completion suggester to give them a search-as-you-type experience.

In this article, we’ll write a completion suggester that both you and the elves can use in your next search use case.

What is a completion suggester?

The ability to type text within a search bar and get back suggestions as you’re typing is something we’re all pretty familiar with at this point. But how exactly is this accomplished within Elasticsearch? Well, Elasticsearch has many APIs that can be used to build a powerful search experience for users. One API in particular, the suggester API, has different types of suggesters whose job is to suggest terms based on a provided text.

In our case, we would be using the completion suggester. A completion suggester is a navigational feature to guide users to relevant results as they are typing, also known as auto-complete or search-as-you-type.

Since the auto-complete functionality should be as fast as a user types the completion suggester is optimized for speed. So as the user types or deletes text, the suggestions will be modified to mirror this experience.

Now that we have a better understanding of what a completion suggester does, let’s take a look at the key components involved in this process.

Major components of a completion suggester

Mapping

In order to implement a completion suggester within your ES cluster, you must define it within your mapping. This will index the field values for fast completions as shown below.

PUT christmas_gift_inventory
{
    "mappings": {
        "properties" : {
            "suggest" : {
                "type" : "completion"
            },
            "christmas_gift" : {
                "type": "keyword"
            }
        }
    }
}

Output

{
  "acknowledged": true,
  "shards_acknowledged": true,
  "index": "christmas_gift_inventory"
}

So here we have a new index called christmas_gift_inventory that will contain all the gifts for each child. Now let's check the mapping to ensure our field has been configured to support fast completion.

GET christmas_gift_inventory/_mapping

Output:

{
  "christmas_gift_inventory": {
    "mappings": {
      "properties": {
        "christmas_gift": {
          "type": "keyword"
        },
        "suggest": {
          "type": "completion",
          "analyzer": "simple",
          "preserve_separators": true,
          "preserve_position_increments": true,
          "max_input_length": 50
        }
      }
    }
  }
}

As you can see the suggester mapping is created with support for the following parameters:

• Analyzer or index analyzer - analyzer specified at index creation

• search analyzer - analyzer to be specified within search query

• Preserve_separators - If set to false, ignores spaces between words when suggesting possible results. So if you suggest foof, then Foo Fighters could be in the field returned back.

• Preserve_position_increments - Enables position increments, defaults to true

• Max_input_length - Limits the length of a single input

After creating the mapping for the index, we are ready to start adding data to our index through a process called indexing.

Indexing

Now that we have our mapping, let’s store the following documents for our completion suggester.

Document 1:

PUT christmas_gift_inventory/_doc/1
{
  "suggest": [
    {
      "input": "Minecraft",
      "weight": 10
    },
    {
      "input": "Mario",
      "weight": 3
    }
  ]
}

Document 2:

PUT christmas_gift_inventory/_doc/2
{
  "suggest": [
    {
      "input": "Midnight Landing",
      "weight": 10
    },
    {
      "input": "Meta Fox",
      "weight": 3
    }
  ]
}

Document 3:

PUT christmas_gift_inventory/_doc/3
{
  "suggest": [
    {
      "input": "Minesweeper",
      "weight": 10
    },
    {
      "input": "Minivader",
      "weight": 3
    }
  ]
}

Document 4:

PUT christmas_gift_inventory/_doc/4
{
  "suggest": [
    {
      "input": "Minesweeper",
      "weight": 10
    },
    {
      "input": "Munchkins",
      "weight": 3
    }
  ]
}

As you can see, we have two suggestions within each document that can be offered up as a suggestion for the user. The suggest field has two parameters: input and weight.

The input field holds the term or suggestion itself and the weight parameter sets how this suggestion should be ranked in comparison to the other terms in the suggestion field. The higher the ranking, the more likely it will be chosen as a possible suggestion for the user's search.

Querying

Here’s a video walkthrough for running queries using the above completion suggester we just created.

Completion Suggester Tutorial

Drawbacks

• Filters can’t be applied
• The suggester uses data structures that enable fast lookups, but are costly to build and are stored in memory.

Conclusion

The completion suggester can be a useful tool for creating an auto-complete functionality within your search engine. This is made possible through the use of data structures that allow for terms to be suggested in real-time.

Once mapping is defined and data is indexed, you will be able to run different queries that allow for suggested results based on provided text. The use of auto-completion at the query level also allows for regex and fuzziness to be used to broaden suggestions while maintaining precision.

Like other autocompletion strategies, the completion suggester has a few drawbacks, including the cost of memory needed to enable fast lookup. Feel free to try this out on your own or in your next search use case.