# Issues with elasticsearch + autocomplete **URL:** \ **Category:** Elasticsearch\ **Created:** [August 16, 2012, 3:08pm UTC](https://discuss.elastic.co/t/issues-with-elasticsearch-autocomplete/8760 "2012-08-16T15:08:23Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Niranjan](https://avatars.discourse-cdn.com/v4/letter/n/e99b99/32.png) [@Niranjan](https://discuss.elastic.co/u/Niranjan)\ **Post date:** [August 16, 2012, 3:08pm UTC](https://discuss.elastic.co/t/issues-with-elasticsearch-autocomplete/8760/1 "2012-08-16T15:08:23Z")
Hello everyone, I have implemented an autocomplete feature to my search input box and am using ngram filter for this. My index\_analyzer is ngram and search\_analyzer is standard. Along with this, I also use highlight, so that the matches are highlighted in a different color in the autocomplete. I have this problem. I have these 2 sentences I'm searching against - "Play baseball", "Play tennis". When I type "P", "Pl", "Pla", the 2 sentences get matched. But when I type "Play", I get 0 matches. If I continue typing to "Play B", then onwards it rightly matches "Play baseball". Why is this so? Another problem- I have a word I'm searching against "Trekking". When I type the trekking, it rightly matches the word but only 'trek' is highlighted and it ignores to highlight 'ing'. eg. when I type 'trek', 'trekk', 'trekki', 'trekkin' the word is rightly matched and it rightly highlights the letters I have typed. When I type 'trekking', only 'trek' is highlighted. Why is this? These issues are driving me up the wall ☹ Regards -- ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [August 16, 2012, 7:28pm UTC](https://discuss.elastic.co/t/issues-with-elasticsearch-autocomplete/8760/2 "2012-08-16T19:28:36Z")
Hi Niranjan > I have implemented an autocomplete feature to my search input box and > am using ngram filter for this. > My index\_analyzer is ngram and search\_analyzer is standard. Along with > this, I also use highlight, so that the matches are highlighted in a > different color in the autocomplete. > > I have this problem. I have these 2 sentences I'm searching against - > "Play baseball", "Play tennis". When I type "P", "Pl", "Pla", the 2 > sentences get matched. But when I type "Play", I get 0 matches. If I > continue typing to "Play B", then onwards it rightly matches "Play > baseball". It's much better to gist a curl recreation of the problem so that we can see exactly what you're doing, and what you could change. With the info we have so far, we'd just be guessing. See [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help) for advice about how gist a recreation clint -- ---
**Author:** ![Niranjan](https://avatars.discourse-cdn.com/v4/letter/n/e99b99/32.png) [@Niranjan](https://discuss.elastic.co/u/Niranjan)\ **Post date:** [August 16, 2012, 7:50pm UTC](https://discuss.elastic.co/t/issues-with-elasticsearch-autocomplete/8760/3 "2012-08-16T19:50:14Z")
Hi Clinton, Thanks for the tip. My gist recreation is here: [gist:3372990 · GitHub](https://gist.github.com/3372990) My mapping is here: [gist:3373031 · GitHub](https://gist.github.com/3373031) Regards, Niranjan On 17 August 2012 00:58, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com) wrote: > Hi Niranjan > > > I have implemented an autocomplete feature to my search input box and > > am using ngram filter for this. > > My index\_analyzer is ngram and search\_analyzer is standard. Along with > > this, I also use highlight, so that the matches are highlighted in a > > different color in the autocomplete. > > > > I have this problem. I have these 2 sentences I'm searching against - > > "Play baseball", "Play tennis". When I type "P", "Pl", "Pla", the 2 > > sentences get matched. But when I type "Play", I get 0 matches. If I > > continue typing to "Play B", then onwards it rightly matches "Play > > baseball". > > It's much better to gist a curl recreation of the problem so that we can > see exactly what you're doing, and what you could change. With the info > we have so far, we'd just be guessing. > > See [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help) for advice about how gist a > recreation > > clint > > -- -- ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [August 16, 2012, 8:40pm UTC](https://discuss.elastic.co/t/issues-with-elasticsearch-autocomplete/8760/4 "2012-08-16T20:40:03Z")
On Fri, 2012-08-17 at 01:20 +0530, Niranjan U wrote: > Hi Clinton, > > Thanks for the tip. > My gist recreation is here: [gist:3372990 · GitHub](https://gist.github.com/3372990) Ummm... You've gisted your cluster health... When I say "gist a curl recreation" I mean something that we can copy and paste and so recreate your problem clint -- ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [August 16, 2012, 8:52pm UTC](https://discuss.elastic.co/t/issues-with-elasticsearch-autocomplete/8760/5 "2012-08-16T20:52:30Z")
Hi Clint. Problem is that there is an example in help page: curl -XGET '[http://127.0.0.1:9200/\_cluster/health?pretty=true](http://127.0.0.1:9200/_cluster/health?pretty=true)' Some users think that they need to give the cluster health. We should enhance this part of the help page and tell users exactly what you wrote: "something that we can copy and paste and so recreate your problem" I think we should add a link to a Gist example with full script: Delete index Create index Index some docs Refresh indice Query What do you think? I can work on that. David -- Le 16 août 2012 à 22:40, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com) a écrit : > On Fri, 2012-08-17 at 01:20 +0530, Niranjan U wrote: > > > Hi Clinton, > > > > Thanks for the tip. > > My gist recreation is here: [gist:3372990 · GitHub](https://gist.github.com/3372990) > > Ummm... You've gisted your cluster health... > > When I say "gist a curl recreation" I mean something that we can copy > and paste and so recreate your problem > > clint > > -- -- ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [August 16, 2012, 9:01pm UTC](https://discuss.elastic.co/t/issues-with-elasticsearch-autocomplete/8760/6 "2012-08-16T21:01:18Z")
Hi David > Problem is that there is an example in help page: curl -XGET > '[http://127.0.0.1:9200/\_cluster/health?pretty=true](http://127.0.0.1:9200/_cluster/health?pretty=true)' > > Some users think that they need to give the cluster health. heh ok I wrote that page a long time ago, and haven't looked at it since. I've just assumed that I expressed myself clearly the first time. Clearly not 🙂 > We should enhance this part of the help page and tell users exactly > what you wrote: "something that we can copy and paste and so recreate > your problem" Indeed! > I think we should add a link to a Gist example with full script: > Delete index > Create index > Index some docs > Refresh indice > Query > > What do you think? I can work on that. That'd be great. thanks clint -- ---
**Author:** ![Niranjan](https://avatars.discourse-cdn.com/v4/letter/n/e99b99/32.png) [@Niranjan](https://discuss.elastic.co/u/Niranjan)\ **Post date:** [August 16, 2012, 11:09pm UTC](https://discuss.elastic.co/t/issues-with-elasticsearch-autocomplete/8760/7 "2012-08-16T23:09:49Z")
Hi Clinton, Sorry about that. I am sending gist of some of the curl operations. Hope this will shed more light on the issue I am facing. 1. Firstly, here is the gist of the mapping I am using: [gist:3373031 · GitHub](https://gist.github.com/3373031) As you can see, my index type "interests" has ngram as my index\_analyzer and "standard" as search\_analyzer. I am also adding porter stem filter to "my\_ngram" analyzer to be able to search over stem words 2. Here are some gist recreations of the issues I am facing: highlight doesn't work on the text: Gist: [gist:3374359 · GitHub](https://gist.github.com/3374359) ``` Now, weird part is, when I type the full word "entrepreneurs", ``` I get 0 results!! [gist:3374366 · GitHub](https://gist.github.com/3374366) I am totally stumped by the kind of results I am getting, any suggestions please? Regards, Niranjan On 17 August 2012 02:31, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com) wrote: > Hi David > > > Problem is that there is an example in help page: curl -XGET > > '[http://127.0.0.1:9200/\_cluster/health?pretty=true](http://127.0.0.1:9200/_cluster/health?pretty=true)' > > > > Some users think that they need to give the cluster health. > > heh ok > > I wrote that page a long time ago, and haven't looked at it since. I've > just assumed that I expressed myself clearly the first time. Clearly > not 🙂 > > > We should enhance this part of the help page and tell users exactly > > what you wrote: "something that we can copy and paste and so recreate > > your problem" > > Indeed! > > > I think we should add a link to a Gist example with full script: > > Delete index > > Create index > > Index some docs > > Refresh indice > > Query > > > > What do you think? I can work on that. > > That'd be great. thanks > > clint > > -- -- ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [August 17, 2012, 9:36am UTC](https://discuss.elastic.co/t/issues-with-elasticsearch-autocomplete/8760/8 "2012-08-17T09:36:50Z")
Hi Niranjan > Sorry about that. I am sending gist of some of the curl operations. > Hope this will shed more light on the issue I am facing. Much better! > 1. Firstly, here is the gist of the mapping I am using: > [gist:3373031 · GitHub](https://gist.github.com/3373031) > As you can see, my index type "interests" has ngram as my > index\_analyzer and "standard" as search\_analyzer. I am also adding > porter stem filter to "my\_ngram" analyzer to be able to search over > stem words OK - there are several things wrong with this mapping: 1. you are using a blunderbuss approach - trying to make EVERYTHING autocomplete and stemmed and and. Rather enable these things selectively, where you really need it 2. Rather use edge-ngrams instead of ngrams. People expect 'tre' to match 'trekking', not 'contretemps'. 3. There is no need to combine porter-stem with ngrams. Porter-stem may convert (eg) 'camping' to 'camp'. Edge-ngrams would give you: c, ca, cam, camp, campi, campin, camping Combined with stemming, you'd just get: c, ca, cam, camp I think that's where your entrpreneurs search is going wrong 4. Use the same analyzer at search and index time, otherwise there is a good chance that you'll be searching for stuff which just isn't there! Also, different analyzers order tokens differently, which can affect results. A better mapping is [gist:992e0e704e035e8a1770 · GitHub](https://gist.github.com/992e0e704e035e8a1770) Note: I'm using a multi-field for "name" with two versions: one analyzed with the 'english' analyzer, and one with edge\_ngrams. The basic query looks like this: curl -XGET '[http://127.0.0.1:9200/test/interests/\_search?pretty=1](http://127.0.0.1:9200/test/interests/_search?pretty=1)' -d ' { "query" : { "bool" : { "should" : [ { "text" : { "name.ngrams" : { "operator" : "and", "query" : "SEARCH TERMS" } } }, { "text" : { "name" : "SEARCH TERMS" } } ] } }, "highlight" : { "fields" : { "name.ngrams" : {}, "name" : {} } } } ' I'm querying the 'name.ngrams' field (which will get the partial words') and I'm also querying the 'name' (or 'name.name') field which will match any full words, and increase their relevance. Note: i'm using the 'and' operator for the ngrams, otherwise 'tre' will match anything that contains just 't'. I'm highlighting both fields. Here are some results: For 'tre': "name.ngrams" : "_tre_kking" For 'trek': ``` "name.ngrams" : "trekking" "name" : "trekking" # note 'trekking' has been highlighted because the stemmed term # is 'trek' ``` For 'bas': "name.ngrams" : "play _bas_ketball" For 'basketbal': ``` "name.ngrams" : "play basketball" "name" : "play basketball" ``` For 'entrepreneurs'; ``` "name.ngrams" : "meet entrepreneurs" "name" : "meet entrepreneurs" # note how the ngram match stops at 10 letters - that's because # your max_gram was set to 10. ``` hth clint -- ---
**Author:** ![Niranjan](https://avatars.discourse-cdn.com/v4/letter/n/e99b99/32.png) [@Niranjan](https://discuss.elastic.co/u/Niranjan)\ **Post date:** [August 18, 2012, 1:04pm UTC](https://discuss.elastic.co/t/issues-with-elasticsearch-autocomplete/8760/9 "2012-08-18T13:04:55Z")
Sincerely appreciate the explanations given and the examples. Got to understand things better! Thanks, this works like a charm! On 17 August 2012 15:06, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com) wrote: > Hi Niranjan > > > Sorry about that. I am sending gist of some of the curl operations. > > Hope this will shed more light on the issue I am facing. > > Much better! > > > 1. Firstly, here is the gist of the mapping I am using: > > [gist:3373031 · GitHub](https://gist.github.com/3373031) > > As you can see, my index type "interests" has ngram as my > > index\_analyzer and "standard" as search\_analyzer. I am also adding > > porter stem filter to "my\_ngram" analyzer to be able to search over > > stem words > > OK - there are several things wrong with this mapping: > > 1. you are using a blunderbuss approach - trying to make EVERYTHING > autocomplete and stemmed and and. Rather enable these things > selectively, where you really need it > > 2. Rather use edge-ngrams instead of ngrams. People expect > 'tre' to match 'trekking', not 'contretemps'. > > 3. There is no need to combine porter-stem with ngrams. > Porter-stem may convert (eg) 'camping' to 'camp'. > Edge-ngrams would give you: > c, ca, cam, camp, campi, campin, camping > Combined with stemming, you'd just get: > c, ca, cam, camp > I think that's where your entrpreneurs search is going wrong > > 4. Use the same analyzer at search and index time, otherwise there > is a good chance that you'll be searching for stuff which just > isn't there! Also, different analyzers order tokens differently, > which can affect results. > > A better mapping is [gist:992e0e704e035e8a1770 · GitHub](https://gist.github.com/992e0e704e035e8a1770) > > Note: I'm using a multi-field for "name" with two versions: one analyzed > with the 'english' analyzer, and one with edge\_ngrams. > > The basic query looks like this: > > curl -XGET '[http://127.0.0.1:9200/test/interests/\_search?pretty=1](http://127.0.0.1:9200/test/interests/_search?pretty=1)' -d ' > { > "query" : { > "bool" : { > "should" : [ > { > "text" : { > "name.ngrams" : { > "operator" : "and", > "query" : "SEARCH TERMS" > } > } > }, > { > "text" : { > "name" : "SEARCH TERMS" > } > } > ] > } > }, > "highlight" : { > "fields" : { > "name.ngrams" : {}, > "name" : {} > } > } > } > ' > > I'm querying the 'name.ngrams' field (which will get the partial words') > and I'm also querying the 'name' (or 'name.name') field which will match > any full words, and increase their relevance. > > Note: i'm using the 'and' operator for the ngrams, otherwise 'tre' will > match anything that contains just 't'. > > I'm highlighting both fields. > > Here are some results: > > For 'tre': > "name.ngrams" : "_tre_kking" > > For 'trek': > > ``` > "name.ngrams" : "trekking" > "name" : "trekking" > > # note 'trekking' has been highlighted because the stemmed term > # is 'trek' > > ``` > > For 'bas': > "name.ngrams" : "play _bas_ketball" > > For 'basketbal': > > ``` > "name.ngrams" : "play basketball" > "name" : "play basketball" > > ``` > > For 'entrepreneurs'; > > ``` > "name.ngrams" : "meet entrepreneurs" > "name" : "meet entrepreneurs" > > # note how the ngram match stops at 10 letters - that's because > # your max_gram was set to 10. > > ``` > > hth > > clint > > -- -- ---
**Author:** ![adhenawer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adhenawer/32/2723_2.png) [@adhenawer](https://discuss.elastic.co/u/adhenawer)\ **Post date:** [September 14, 2012, 7:49pm UTC](https://discuss.elastic.co/t/issues-with-elasticsearch-autocomplete/8760/10 "2012-09-14T19:49:56Z")
Hi, your post was very useful. I wonder of the hightligth of ngrams is applied to compound words, example. " _samsung gala " from "samsung galaxy". I did exactly as shown but took only one word._ _Thanks!_ _Em sexta-feira, 17 de agosto de 2012 06h36min50s UTC-3, Clinton Gormley escreveu:_ _> Hi Niranjan > > > Sorry about that. I am sending gist of some of the curl operations. > > Hope this will shed more light on the issue I am facing. > > Much better! > > > 1. Firstly, here is the gist of the mapping I am using: > > [gist:3373031 · GitHub](https://gist.github.com/3373031) > > As you can see, my index type "interests" has ngram as my > > index\_analyzer and "standard" as search\_analyzer. I am also adding > > porter stem filter to "my\_ngram" analyzer to be able to search over > > stem words > > OK - there are several things wrong with this mapping: > > 1. you are using a blunderbuss approach - trying to make EVERYTHING > autocomplete and stemmed and and. Rather enable these things > selectively, where you really need it > > 2. Rather use edge-ngrams instead of ngrams. People expect > 'tre' to match 'trekking', not 'contretemps'. > > 3. There is no need to combine porter-stem with ngrams. > Porter-stem may convert (eg) 'camping' to 'camp'. > Edge-ngrams would give you: > c, ca, cam, camp, campi, campin, camping > Combined with stemming, you'd just get: > c, ca, cam, camp > I think that's where your entrpreneurs search is going wrong > > 4. Use the same analyzer at search and index time, otherwise there > is a good chance that you'll be searching for stuff which just > isn't there! Also, different analyzers order tokens differently, > which can affect results. > > A better mapping is [gist:992e0e704e035e8a1770 · GitHub](https://gist.github.com/992e0e704e035e8a1770) > > Note: I'm using a multi-field for "name" with two versions: one analyzed > with the 'english' analyzer, and one with edge\_ngrams. > > The basic query looks like this: > > curl -XGET '[http://127.0.0.1:9200/test/interests/\_search?pretty=1](http://127.0.0.1:9200/test/interests/_search?pretty=1)' -d ' > { > "query" : { > "bool" : { > "should" : [ > { > "text" : { > "name.ngrams" : { > "operator" : "and", > "query" : "SEARCH TERMS" > } > } > }, > { > "text" : { > "name" : "SEARCH TERMS" > } > } > ] > } > }, > "highlight" : { > "fields" : { > "name.ngrams" : {}, > "name" : {} > } > } > } > ' > > I'm querying the 'name.ngrams' field (which will get the partial words') > and I'm also querying the 'name' (or 'name.name') field which will match > any full words, and increase their relevance. > > Note: i'm using the 'and' operator for the ngrams, otherwise 'tre' will > match anything that contains just 't'. > > I'm highlighting both fields. > > Here are some results: > > For 'tre': > "name.ngrams" : "trekking" > > For 'trek': > > ``` > "name.ngrams" : "trekking" > "name" : "trekking" > > # note 'trekking' has been highlighted because the stemmed term > # is 'trek' > > ``` > > For 'bas': > "name.ngrams" : "play basketball" > > For 'basketbal': > > ``` > "name.ngrams" : "play basketball" > "name" : "play basketball" > > ``` > > For 'entrepreneurs'; > > ``` > "name.ngrams" : "meet entrepreneurs" > "name" : "meet entrepreneurs" > > # note how the ngram match stops at 10 letters - that's because > # your max_gram was set to 10. > > ``` > > hth > > clint_ _--_ ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:12am UTC](https://discuss.elastic.co/t/issues-with-elasticsearch-autocomplete/8760/11 "2017-07-06T03:12:41Z")