Autocomplete based on past user searches

[abhijith_chandran (Tiations)]

I am trying to add autocomplete based on what the user searches.
Currently, I have the following mapping:

{
  "courts_2": {
    "mappings": {
      "properties": {
        "author": {
          "type": "text",
          "analyzer": "my_analyzer"
        },
        "bench": {
          "type": "text",
          "analyzer": "my_analyzer"
        },
        "citation": {
          "type": "text"
        },
        "content": {
          "type": "text",
          "fields": {
            "standard": {
              "type": "text"
            }
          },
          "analyzer": "my_analyzer"
        },
        "court": {
          "type": "text"
        },
        "date": {
          "type": "text"
        },
        "id_": {
          "type": "text"
        },
        "title": {
          "type": "text",
          "fields": {
            "standard": {
              "type": "text"
            }
          },
          "analyzer": "my_analyzer"
        },
        "verdict": {
          "type": "text"
        }
      }
    }
  }
}

Below is the code I used for the settings:

{
  "settings": {
    "index": {
      "analysis": {
        "analyzer": {
          "my_analyzer": {
            "tokenizer": "standard",
            "filter": [
              "lowercase",
              "my_metaphone"
            ]
          }
        },
        "filter": {
          "my_metaphone": {
            "type": "phonetic",
            "encoder": "metaphone",
            "replace": true
          }
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "author": {
        "type": "text",
        "analyzer": "my_analyzer"
      },
      "bench": {
        "type": "text",
        "analyzer": "my_analyzer"
      },
      "citation": {
        "type": "text"
      },
      "court": {
        "type": "text"
      },
      "date": {
        "type": "text"
      },
      "id_": {
        "type": "text"
      },
      "verdict": {
        "type": "text"
      },
      "title": {
        "type": "text",
        "analyzer": "my_analyzer",
        "fields": {
          "standard": {
            "type": "text"
          }
        }
      },
      "content": {
        "type": "text",
        "analyzer": "my_analyzer",
        "fields": {
          "standard": {
            "type": "text"
          }
        }
      }
    }
  }
}

Here's what I would like to implement: I would like to collect and store all of the queries made to the endpoint and use autocomplete on that. For example, to date, all of the users have made the following queries -

Real Madrid v/s Barcelona 
Real Madrid Team 
Real Madrid Coach 
Barcelona v/s Man City 
Sevilla Home Ground
Man Utd. recent results

Now, if anyone searches Rea then the following autocomplete queries should be suggested:

Real Madrid v/s Barcelona 
Real Madrid Team 
Real Madrid Coach 

This based on the searches made by all of the users till date and not a single user. Further, I would like to analyze what are the top queries that were made in let's say the past month.

I am using ElasticSearch version 7.1 on AWS Elasticsearch service.

[spinscale (Alexander Reelsen)]

This is a complex topic and cannot really be answered in a small quick reply. Just adding a couple of throughts. First you have to think about what kind of suggestions you want

1. If they should be based on your data (and not the queries), then take a look at the search-as-you-type datatype, see https://www.elastic.co/guide/en/elasticsearch/reference/7.x/search-as-you-type.html

2. If you want this to be based on the existing searches, you first need to search log all the searches typed by your users and use that as a data source for suggestions (you could again use the afore mentioned search as you type datatype). However you would still need to ensure that your searches also return documents and probably have some logic to score popular searches higher (i.e. via rank_feature), see https://www.elastic.co/guide/en/elasticsearch/reference/master/rank-feature.html

As you can see, to implement this properly, this requires some more trial and error to come up with a solution. Hope this helps as a first hint.

--Alex

[abhijith_chandran (Tiations)]

I'll go through the links. Thank you, @spinscale!