Completion suggester suggesting full document

Elastic Stack Elasticsearch
1

Hi,

I am migrating from ES 1.7 to 5.2 and as I am testing I noticed that the new completion suggester does not work as it used to in 1.7.

In 1.7 when I invoke the _suggest API the response returned a set of options matching the distinct words/entries found in the completion field.

In 5.2, I am getting the entire document inside the options object.

For example if my completion field has the following array stored:

"input": ["winter", "windows", "widow"]

In 1.7 when I invoke the _suggest API I used to get this response:

{
	"_shards" : {
		"total" : 1,
		"successful" : 1,
		"failed" : 0
	},
	"tagsuggest" : [{
			"text" : "wi",
			"offset" : 0,
			"length" : 2,
			"options" : [{
					"text" : "widow",
					"score" : 3.0
				}, {
					"text" : "windows",
					"score" : 3.0
				}, {
					"text" : "winter",
					"score" : 3.0
				}
			]
		}
	]
}

Yet in 5.2 when I do a suggest search I am getting the full document:

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "hits": {
    "total": 0,
    "max_score": 0,
    "hits": []
  },
  "suggest": {
    "tagssuggest": [
      {
        "text": "wi",
        "offset": 0,
        "length": 3,
        "options": [
          {
            "text": "winning",
            "_index": "myindex",
            "_type": "mytype",
            "_id": "123456",
            "_score": 1,
            "_source": {
              "tags": [
                "windows",
                "winter",
                "winning"
              ]
              "tagssuggest": {
                "input": [
                  "windows",
				  "winter",
                  "winning"
                ],
                "contexts": {
                  "user": [
                    "1"
                  ]
                }
              }
            },
            "contexts": {
              "user": [
                "1"
              ]
            }
          }
        ]
      }
    ]
  }
}

How can I go about returning distinct matching words from the suggest field?

Regards

2

Hi,

Suggesters have changed a lot since 1.7. In 5.x they are now document oriented, ie the completion suggester now return the document from where suggestions have been found (see https://www.elastic.co/guide/en/elasticsearch/reference/5.0/breaking_50_suggester.html).

You might either want to filter the source of the suggested documents (see https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters-completion.html) or use a Term suggester instead (see https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters-term.html)

Hope this helps

3

This is really frustrating and I do not see the need for an auto complete suggester to return an entire document? What was the argument against returning specific field data for an auto complete suggester?

I don't think filtering the source is sufficient since I might have several documents having the same tag. So I might be getting the same suggested tag/element multiple times.

As for the term suggester, I don't see how I can use it as a completion suggester? Well, first I need to apply a query as I do not want to suggest on the entire index.

So that probably leaves me with the Phrase suggester which I am not quite sure how to utilize it as an auto-complete/prefix-based suggester.

Any examples?

4

I think I found the solution however it will require the tags to be stored in two separate fields (one to query and the other to aggregate on)

{
	"from" : 0,
	"size" : 0,
	"query" : {
		"bool" : {
			"must" : [{
					"term" : {
						"userid" : {
							"value" : 1
						}
					}
				}
			]
		}
	},
	"aggs" : {
		"suggest" : {
			"filter" : {
				"prefix" : {
					"tags" : {
						"value" : "win"
					}
				}
			},
			"aggs" : {
				"Tags_suggest" : {
					"terms" : {
						"field" : "tags_key",
						"size" : 20,
						"include" : {
							"pattern" : "win.*"
						}
					}
				}
			}
		}
	}
}

I can live with that, even though it will require twice the storage. And fingers crossed that ES wont remove aggregation filters in the future.

1 Like
5

The same for me - It's a shame that it has become document-oriented.

The suggestions I want my users to see might not have anything to do with specific documents. Suggestions could be based on historic queries, shingles or the like. The old suggester gave us a way to get a list of suggestions matching the prefix of the input string - now we only get documents matching and no other near matches if there isn't a document directly linked to it. At least it would have been nice to have a way to keep the old functionality.

I am now forced to go with a terms aggregation with a regex include filter. This means I have no way of providing fuzzy matches now, which is a shame for our solution.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.