Autocomplete across multiple fields with NEST client

Elastic Stack Elasticsearch
1

Hi!

I`am trying implement autocomplete feature for multiple fields with official high-level NEST client.
But quick and dirty solution looks like this:

        public async Task<IEnumerable<string>> Complete(string text)
    {
        var sd1 = new SearchDescriptor<Assignment>().Suggest(
            scd => scd.Completion(
                "n1",
                csd => csd
                    .Field(x => x.Title)
                    .Prefix(text)));

        var sd2 = new SearchDescriptor<Assignment>().Suggest(
            scd => scd.Completion(
                "n2",
                csd => csd
                    .Field(x => x.Theme)
                    .Prefix(text)));

        var sd3 = new SearchDescriptor<Assignment>().Suggest(
            scd => scd.Completion(
                "n3",
                csd => csd
                    .Field(x => x.TrainingKit)
                    .Prefix(text)));

        var task1 = _elasticClient.SearchAsync<Assignment>(sd1);
        var task2 = _elasticClient.SearchAsync<Assignment>(sd2);
        var task3 = _elasticClient.SearchAsync<Assignment>(sd3);

        await Task.WhenAll(task1, task2, task3);

        var resp1 = task1.Result.Suggest.SelectMany(x => x.Value).SelectMany(x => x.Options).Select(x => x.Text).Distinct();
        var resp2 = task2.Result.Suggest.SelectMany(x => x.Value).SelectMany(x => x.Options).Select(x => x.Text).Distinct();
        var resp3 = task3.Result.Suggest.SelectMany(x => x.Value).SelectMany(x => x.Options).Select(x => x.Text).Distinct();

        return resp1.Concat(resp2).Concat(resp3);
    }

Completion dsl method have only Field option, but not Fields. Can i implement this fearure without call Task.WhenAll(...) with NEST?

2

Here's an example of supplying multiple inputs to Completion Suggester

void Main()
{
	var index = "assignments";
	var settings = new ConnectionSettings(new Uri("http://localhost:9200"))
		.DefaultIndex(index);

	var client = new ElasticClient(settings);

	if (client.IndexExists(index).Exists)
	{
		client.DeleteIndex(index);
	}
	
	client.CreateIndex(index, c => c
		.Mappings(m => m
			.Map<Assignment>(mm => mm
				.AutoMap()
			)
		)
	);

	client.IndexMany(new[]
		{
			new Assignment
			{
				Id = 1,
				Title = "An assignment about Elasticsearch",
				Theme = "Data analytics",
				TrainingKit = "Developer course",
				Suggest = new CompletionField
				{
					Input = new []
					{
						"An assignment about Elasticsearch",
						"Data analytics",
						"Developer course",
					},
					Weight = 100
				}
			},
			new Assignment
			{
				Id = 2,
				Title = "All about Kibana",
				Theme = "Diagrams and Graphs",
				TrainingKit = "Youtube videos",
				Suggest = new CompletionField
				{
					Input = new []
					{
						"All about Kibana",
						"Diagrams and Graphs",
						"Youtube videos",
					},
					Weight = 80
				}
			},
		});
		
	client.Refresh(index);
	
	var input = "d";

	var response = client.Search<Assignment>(s => s
		.Index<Assignment>()
        .Source(so => so
            .Includes(f => f
                .Field(ff => ff.Id)
            )
        )
        .Suggest(su => su
    		.Completion("suggest", cs => cs
    			.Field(f => f.Suggest)
    			.Prefix(input)
    			.Fuzzy(f => f
    				.Fuzziness(Fuzziness.Auto)
    			)
    			.Size(10)
    		)
        )
    );

    // do something with the suggestions
    var suggestions =
        from suggest in response.Suggest["suggest"]
        from option in suggest.Options
        select new
        {
            Id = option.Source.Id,
            Text = option.Text,
            Votes = option.Score
        };


}

public class Assignment
{
	public int Id { get;set; }	
	public string Title { get; set; }	
	public string Theme { get;set; }	
	public string TrainingKit { get;set; }
	public CompletionField Suggest { get; set; }	
}

Some important points:

  1. Completion suggester works with the completion data type field mapping. With NEST's automapping, Nest.CompletionField will be mapped as completion field
  2. CompletionField's Input property takes an IEnumerable<string> to allow multiple inputs
  3. Use "suggest" on the _search API to make the call, targeting the Suggest property of the POCO, which will map to the "suggest" field in Elasticsearch
  4. Completion Suggester is designed for fast "search as you type". If your search needs are more complex, you may need to build custom analyzers and take a multi-faceted approach to search.

The response from the search request looks like

{
  "took" : 28,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 0,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "suggest" : {
    "completion#suggest" : [
      {
        "text" : "d",
        "offset" : 0,
        "length" : 1,
        "options" : [
          {
            "text" : "Data analytics",
            "_index" : "assignments",
            "_type" : "assignment",
            "_id" : "1",
            "_score" : 100.0,
            "_source" : {
              "id" : 1
            }
          },
          {
            "text" : "Diagrams and Graphs",
            "_index" : "assignments",
            "_type" : "assignment",
            "_id" : "2",
            "_score" : 80.0,
            "_source" : {
              "id" : 2
            }
          }
        ]
      }
    ]
  }
}
3

:pray: Thanks, very helpfull example!

One more common question about autocomplete. For example, completions must appear ahead based on some query statistics over the last week. I'm not sure how correctly implement this feature. Should i use aggregation for this or other functionality and how elastic could tracking query statistics?

4

Completion suggester may not be the best API to use for this. Take a look at function_score query with decay function on the document field that represents the date of interest.

Elasticsearch does not currently have a feature that captures data about the search queries that are executed, so you would need to implement something to fit your needs here. This may be as simple as capturing data such as:

  1. The terms searched for
  2. The number of searches performed before result is selected (or abandoned)
  3. The position of the selected result (if one is selected)
5

A bit more about your code sample. If user type "about Elasticsearch..." the string "An assignment about Elasticsearch" not suggested, becuse elasticsearch provide only prefix suggestions. Could i use for this any standart suggester or i need write own java plugin for elasticsearch?

For inputs, "foo", "bar", "baz", "foo bar baz", "foo bar", "bar baz", "foo baz" my application should suggest string "foo bar baz";

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.