# How highlighting actually works?

**URL:** <https://discuss.elastic.co/t/how-highlighting-actually-works/21703>\
**Category:** Elasticsearch\
**Created:** [January 18, 2015, 11:21pm UTC](https://discuss.elastic.co/t/how-highlighting-actually-works/21703 "2015-01-18T23:21:33Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Karol\_Sikora](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karol_sikora/32/983_2.png) [@Karol\_Sikora](https://discuss.elastic.co/u/Karol_Sikora)\
**Post date:** [January 18, 2015, 11:21pm UTC](https://discuss.elastic.co/t/how-highlighting-actually-works/21703/1 "2015-01-18T23:21:33Z")

</div>

Hi all,

I have some specific requirements for highlighting. I need to search in  
full content of item for phrase, and then show on which page searched  
phrase is occuring. So i've created one field named text\_content and fields  
named text\_content\_{page\_number} (text\_content\_1, text\_content\_2, etc.).  
Example query is:  
{  
"highlight": {  
"fields": {  
"text\_content\_\*": {}  
}  
},  
"query": {  
"match": {  
"text\_content": "lorem"  
}  
},  
"size": 40  
}

I've noticed that this query is fast, but only if i have small number of  
documents in index. Quiering for documents is always fast (\<40ms), but  
highlight phase time is growing when number of documents in index is  
growing.  
I've stared thinking that highlighting may be processed before appending  
"size": 40 - on the all matched documents. It's correct? How can in speed  
up such case?

Regards,  
Karol

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b8354eb3-3a75-4999-a180-6493240eb0cc%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b8354eb3-3a75-4999-a180-6493240eb0cc%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [January 19, 2015, 12:31am UTC](https://discuss.elastic.co/t/how-highlighting-actually-works/21703/2 "2015-01-19T00:31:15Z")

</div>

Highlighting is complex and more hacky than you'd imagine at first glance.  
Each highlighter is different and we can't tell which one you are using  
without seeing your mapping. For the plain highlighter the cost is roughly  
proportional to the length of the highlighted field. So in your case its  
the cost to reanalyze every one of those pages.

You could return which page is matched pretty cheaply if you were willing  
to write a plugin. Especially if you just wanted to know the first page or  
something.

You could try using explain if you searched for text\_content\_\*. That'd  
tell you which field matched.

Nik  
On Jan 18, 2015 6:21 PM, "Karol Sikora" [sicarrots@gmail.com](mailto:sicarrots@gmail.com) wrote:

> Hi all,
> 
> I have some specific requirements for highlighting. I need to search in  
> full content of item for phrase, and then show on which page searched  
> phrase is occuring. So i've created one field named text\_content and fields  
> named text\_content\_{page\_number} (text\_content\_1, text\_content\_2, etc.).  
> Example query is:  
> {  
> "highlight": {  
> "fields": {  
> "text\_content\_\*": {}  
> }  
> },  
> "query": {  
> "match": {  
> "text\_content": "lorem"  
> }  
> },  
> "size": 40  
> }
> 
> I've noticed that this query is fast, but only if i have small number of  
> documents in index. Quiering for documents is always fast (\<40ms), but  
> highlight phase time is growing when number of documents in index is  
> growing.  
> I've stared thinking that highlighting may be processed before appending  
> "size": 40 - on the all matched documents. It's correct? How can in speed  
> up such case?
> 
> Regards,  
> Karol
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/b8354eb3-3a75-4999-a180-6493240eb0cc%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b8354eb3-3a75-4999-a180-6493240eb0cc%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/b8354eb3-3a75-4999-a180-6493240eb0cc%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/b8354eb3-3a75-4999-a180-6493240eb0cc%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd221YctsJE3QrkqnffjXACNzcZ5WaiuR1Ucrr0DV\_U\_NA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd221YctsJE3QrkqnffjXACNzcZ5WaiuR1Ucrr0DV_U_NA%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Karol\_Sikora](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karol_sikora/32/983_2.png) [@Karol\_Sikora](https://discuss.elastic.co/u/Karol_Sikora)\
**Post date:** [January 19, 2015, 3:37pm UTC](https://discuss.elastic.co/t/how-highlighting-actually-works/21703/3 "2015-01-19T15:37:59Z")

</div>

Thank for your answer. I'm probably posted too few details, here is better  
description:

I'm using postings highlighter, but also checked plain and fvh - both was  
remarkable slower in my case.  
Fields text\_content\* are mapped through dynamic template:  
"dynamic\_templates": [  
{  
"text\_content": {  
"match": "text\_content\*",  
"match\_mapping\_type": "string",  
"mapping": {  
"type": "string",  
"analyzer": "polish",  
"index\_options": "offsets"  
}  
}  
}  
]  
}

polish analyzer is defined as follow (using this plugin:  
[GitHub - monterail/elasticsearch-analysis-morfologik: Morfologik (Polish) Analysis Plugin for ElasticSearch](https://github.com/monterail/elasticsearch-analysis-morfologik) which  
provides morfologik\_stem token filter):  
"analyzer": {  
"polish": {  
"type": "custom",  
"tokenizer": "standard",  
"filter": [  
"lowercase",  
"morfologik\_stem"  
]  
}  
}  
Pure quering in text\_content is always very fast - tooks \<40ms.  
Total amount of time for executing request is increasing when number of  
matched documents grows (more items added to index).  
So i've stared thinking that highlighter is working for all of matched  
documents, not only for items requested by current request (start and size  
parameters). It's correct? There is some way to speed up such case (forcing  
to highlight only in requested window of documents?).

Karol

2015-01-19 1:31 GMT+01:00 Nikolas Everett [nik9000@gmail.com](mailto:nik9000@gmail.com):

> Highlighting is complex and more hacky than you'd imagine at first glance.  
> Each highlighter is different and we can't tell which one you are using  
> without seeing your mapping. For the plain highlighter the cost is roughly  
> proportional to the length of the highlighted field. So in your case its  
> the cost to reanalyze every one of those pages.
> 
> You could return which page is matched pretty cheaply if you were willing  
> to write a plugin. Especially if you just wanted to know the first page or  
> something.
> 
> You could try using explain if you searched for text\_content\_\*. That'd  
> tell you which field matched.
> 
> Nik  
> On Jan 18, 2015 6:21 PM, "Karol Sikora" [sicarrots@gmail.com](mailto:sicarrots@gmail.com) wrote:
> 
> > Hi all,
> > 
> > I have some specific requirements for highlighting. I need to search in  
> > full content of item for phrase, and then show on which page searched  
> > phrase is occuring. So i've created one field named text\_content and fields  
> > named text\_content\_{page\_number} (text\_content\_1, text\_content\_2, etc.).  
> > Example query is:  
> > {  
> > "highlight": {  
> > "fields": {  
> > "text\_content\_\*": {}  
> > }  
> > },  
> > "query": {  
> > "match": {  
> > "text\_content": "lorem"  
> > }  
> > },  
> > "size": 40  
> > }
> > 
> > I've noticed that this query is fast, but only if i have small number of  
> > documents in index. Quiering for documents is always fast (\<40ms), but  
> > highlight phase time is growing when number of documents in index is  
> > growing.  
> > I've stared thinking that highlighting may be processed before appending  
> > "size": 40 - on the all matched documents. It's correct? How can in speed  
> > up such case?
> > 
> > Regards,  
> > Karol
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/b8354eb3-3a75-4999-a180-6493240eb0cc%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b8354eb3-3a75-4999-a180-6493240eb0cc%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/b8354eb3-3a75-4999-a180-6493240eb0cc%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/b8354eb3-3a75-4999-a180-6493240eb0cc%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .  
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> You received this message because you are subscribed to a topic in the  
> Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit  
> [https://groups.google.com/d/topic/elasticsearch/FzSTLVWyok8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/FzSTLVWyok8/unsubscribe).  
> To unsubscribe from this group and all its topics, send an email to  
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd221YctsJE3QrkqnffjXACNzcZ5WaiuR1Ucrr0DV\_U\_NA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd221YctsJE3QrkqnffjXACNzcZ5WaiuR1Ucrr0DV_U_NA%40mail.gmail.com)  
> [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd221YctsJE3QrkqnffjXACNzcZ5WaiuR1Ucrr0DV\_U\_NA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd221YctsJE3QrkqnffjXACNzcZ5WaiuR1Ucrr0DV_U_NA%40mail.gmail.com?utm_medium=email&utm_source=footer)  
> .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAN8rAyJC38RPkxZTxb0tvX1UcsW4mtO\_f1tBRrpQ3ssSQQaXHA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAN8rAyJC38RPkxZTxb0tvX1UcsW4mtO_f1tBRrpQ3ssSQQaXHA%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:37am UTC](https://discuss.elastic.co/t/how-highlighting-actually-works/21703/4 "2017-07-06T00:37:58Z")

</div>