# Highlighting not working for \[edge\_\]ngram with the new versions

**URL:** <https://discuss.elastic.co/t/highlighting-not-working-for-edge--ngram-with-the-new-versions/14467>\
**Category:** Elasticsearch\
**Created:** [November 19, 2013, 10:52am UTC](https://discuss.elastic.co/t/highlighting-not-working-for-edge--ngram-with-the-new-versions/14467 "2013-11-19T10:52:42Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Guillermo\_Arias\_del\_](https://avatars.discourse-cdn.com/v4/letter/g/cab0a1/32.png) [@Guillermo\_Arias\_del\_](https://discuss.elastic.co/u/Guillermo_Arias_del_)\
**Post date:** [November 19, 2013, 10:52am UTC](https://discuss.elastic.co/t/highlighting-not-working-for-edge--ngram-with-the-new-versions/14467/1 "2013-11-19T10:52:42Z")

</div>

Hi, all!

We recently updated from 0.90.1 to 0.90.6 and our highlighting tests began  
to fail. Updating to 0.90.7 didn't work, so I think there is a bug, or at  
least something changed in the specification...

I added a Gist to reproduce the problem:

> <https://gist.github.com/ariasdelrio/7543562>

Here, I am searching for "te" in a field containing "some text to  
highlight". The field is folded and tokenized using the ICU plug-in. The  
value I expect (and got in earlier versions) is _"some text to  
highlight"_, but the returned value is _"some text to  
highlight"_. I checked with the three highlighters (I only need the  
postings highlighter, but I thought I should check with the others as well).

Is this a bug or am I missing something?

Thanks,  
Guillermo.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Guillermo\_Arias\_del\_](https://avatars.discourse-cdn.com/v4/letter/g/cab0a1/32.png) [@Guillermo\_Arias\_del\_](https://discuss.elastic.co/u/Guillermo_Arias_del_)\
**Post date:** [November 19, 2013, 1:18pm UTC](https://discuss.elastic.co/t/highlighting-not-working-for-edge--ngram-with-the-new-versions/14467/2 "2013-11-19T13:18:38Z")

</div>

I found an answer to this:

> <https://github.com/elastic/elasticsearch/issues/3137>
>
> when using an nGram filter on a field or index, if you try to highlight said fie…ld (or a field in an index that has an nGram filter defined on it, in search results, elasticsearch highlights the entire word instead of just the query.
> 
> so if I have the text "American" and I search for "rican"
> highlighting should look like this ----\> Ame \*\*rican\*\*
> but instead it does this ---\> \*\*American\*\*
> 
> To see this in action just follow the instructions here http://stackoverflow.com/a/15005321/141822
> you get this output, which is clearly wrong
> 
> \`\`\`
> {
> "took" : 11,
> "timed\_out" : false,
> "\_shards" : {
> "total" : 1,
> "successful" : 1,
> "failed" : 0
> },
> "hits" : {
> "total" : 3,
> "max\_score" : 0.71231794,
> "hits" : \[ {
> "\_index" : "myindex",
> "\_type" : "product",
> "\_id" : "0KyaIB8xRmqE-g0hl0ky6g",
> "\_score" : 0.71231794,
> "fields" : {
> "code" : "Samsung Galaxy i7500"
> },
> "highlight" : {
> "code.ngram" : \["\<em\>Samsung Galaxy i7500\</em\>" \],
> "code" : \["\<em\>Samsung Galaxy i7500\</em\>" \]
> }
> }, {
> "\_index" : "myindex",
> "\_type" : "product",
> "\_id" : "vZwpcBu0QAyGmP9LHz1hUA",
> "\_score" : 0.71231794,
> "fields" : {
> "code" : "Samsung Galaxy 5 Europa"
> },
> "highlight" : {
> "code.ngram" : \["\<em\>Samsung Galaxy 5 Europa\</em\>" \],
> "code" : \["\<em\>Samsung Galaxy 5 Europa\</em\>" \]
> }
> }, {
> "\_index" : "myindex",
> "\_type" : "product",
> "\_id" : "7sNkZAlxSlmuLZA9S68bvg",
> "\_score" : 0.71231794,
> "fields" : {
> "code" : "Samsung Galaxy Mini"
> },
> "highlight" : {
> "code.ngram" : \["\<em\>Samsung Galaxy Mini\</em\>" \],
> "code" : \["\<em\>Samsung Galaxy Mini\</em\>" \]
> }
> } \]
> }
> }
> \`\`\`
> 
> With the whitespace tokenizer (vs keyword tokenizer in this case), it highlights just the word with the match in it, which is still not expected behavior

Now, I'm using the "version": "4.1", though it may cause problems, as it is  
written in the post. Also, I forgot to specify a different  
"search\_analyzer" in my Gist, which explains the match against "to". I  
tried to get an example with a minimal configuration and cut that off 🙂

I think, however, that the documentation should have a few lines explaining  
what you can and cannot expect from highlighting, because it can drive you  
crazy.

El martes, 19 de noviembre de 2013 11:52:42 UTC+1, Guillermo Arias del Río  
escribió:

> Hi, all!
> 
> We recently updated from 0.90.1 to 0.90.6 and our highlighting tests began  
> to fail. Updating to 0.90.7 didn't work, so I think there is a bug, or at  
> least something changed in the specification...
> 
> I added a Gist to reproduce the problem:  
> [Highlighting not working in ES 0.90.6/7 · GitHub](https://gist.github.com/ariasdelrio/7543562)
> 
> Here, I am searching for "te" in a field containing "some text to  
> highlight". The field is folded and tokenized using the ICU plug-in. The  
> value I expect (and got in earlier versions) is _"some text to  
> highlight"_, but the returned value is _"some text to  
> highlight"_. I checked with the three highlighters (I only need the  
> postings highlighter, but I thought I should check with the others as well).
> 
> Is this a bug or am I missing something?
> 
> Thanks,  
> Guillermo.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![javanna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javanna/32/4698_2.png) [@javanna](https://discuss.elastic.co/u/javanna)\
**Post date:** [November 20, 2013, 12:58pm UTC](https://discuss.elastic.co/t/highlighting-not-working-for-edge--ngram-with-the-new-versions/14467/3 "2013-11-20T12:58:37Z")

</div>

Hi Guillermo,  
are you sure highlighting was working properly with the same mapping before?

What I see in your gist is that you use the edge ngram filter, the default  
one available which unfortunately has min\_gram 1 and max\_gram 2 . Have a  
look at the analyze api output to see what you are  
indexing: [http://localhost:9200/test\_highlight/\_analyze?analyzer=icu\_analyzer&text=some%20text%20to%20highlight](http://localhost:9200/test_highlight/_analyze?analyzer=icu_analyzer&text=some%20text%20to%20highlight)  
.

"Some text to highlight" becomes s, t, t, h . At query time, te becomes t  
as well (as you apply the same analyzer at search time too), which is why  
you get the second and third token highlighted, makes sense to me.

On Tuesday, November 19, 2013 2:18:38 PM UTC+1, Guillermo Arias del Río  
wrote:

> I found an answer to this:  
> [elasticsearch highlights entire word instead of just the query when ngram filter is used · Issue #3137 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/3137)  
> Now, I'm using the "version": "4.1", though it may cause problems, as it  
> is written in the post. Also, I forgot to specify a different  
> "search\_analyzer" in my Gist, which explains the match against "to". I  
> tried to get an example with a minimal configuration and cut that off 🙂
> 
> I think, however, that the documentation should have a few lines  
> explaining what you can and cannot expect from highlighting, because it can  
> drive you crazy.
> 
> El martes, 19 de noviembre de 2013 11:52:42 UTC+1, Guillermo Arias del Río  
> escribió:
> 
> > Hi, all!
> > 
> > We recently updated from 0.90.1 to 0.90.6 and our highlighting tests  
> > began to fail. Updating to 0.90.7 didn't work, so I think there is a bug,  
> > or at least something changed in the specification...
> > 
> > I added a Gist to reproduce the problem:  
> > [Highlighting not working in ES 0.90.6/7 · GitHub](https://gist.github.com/ariasdelrio/7543562)
> > 
> > Here, I am searching for "te" in a field containing "some text to  
> > highlight". The field is folded and tokenized using the ICU plug-in. The  
> > value I expect (and got in earlier versions) is _"some text to  
> > highlight"_, but the returned value is _"some text to  
> > highlight"_. I checked with the three highlighters (I only need the  
> > postings highlighter, but I thought I should check with the others as well).
> > 
> > Is this a bug or am I missing something?
> > 
> > Thanks,  
> > Guillermo.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:05am UTC](https://discuss.elastic.co/t/highlighting-not-working-for-edge--ngram-with-the-new-versions/14467/4 "2017-07-06T02:05:57Z")

</div>