# Highlight problem **URL:** https://discuss.elastic.co/t/highlight-problem/3692 **Category:** Elasticsearch **Created:** [December 22, 2010, 7:29pm UTC](https://discuss.elastic.co/t/highlight-problem/3692 "2010-12-22T19:29:02Z") **Posts on this page:** 8 **Page:** 1
### Author: ![locojay](https://avatars.discourse-cdn.com/v4/letter/l/838e76/32.png) [@locojay](https://discuss.elastic.co/u/locojay) #### Post date: [December 22, 2010, 7:29pm UTC](https://discuss.elastic.co/t/highlight-problem/3692/1 "2010-12-22T19:29:02Z")
Hi my mapping is "mongodocid":{"boost":1.0, "store": "yes", "type": "string"}, "fulltext": {"boost": 1.0, "index": "analyzed", "store": "yes", "type": "string", "term\_vector" : "with\_positions\_offsets"}} to highlight the complete fulltext i am setting number\_of\_framgments to 0. if i do the following lucene like string query: '{"highlight": {"pre\_tags": "**", "fields": {"fulltext": {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": {"query\_string": {"query": "fulltext:test"}}, "size": 100}' for some documents in the resulset the lengh of the highlighted fulltext is smaller than ten fulltext itself. Since i am setting number\_of\_fragments to 0 and pre\_tags/post\_tags are added this should not happen. Now comes the strange behaviour. if i only search for one of this failing elements by doing this {"highlight": {"pre\_tags": "**", "fields": {"fulltext": {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": {"query\_string": {"query": "fulltext:test AND mongodocid: 4d0a861c2ebef6032c00b1ec"}}, "size": 100} then all works fine. Any idea? ---
### Author: ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1) #### Post date: [December 22, 2010, 7:36pm UTC](https://discuss.elastic.co/t/highlight-problem/3692/2 "2010-12-22T19:36:49Z")
Hi, which version of ES are you using? Regards, Lukas Dne 22.12.2010 20:29 "locojay" [jassin.meknassi@googlemail.com](mailto:jassin.meknassi@googlemail.com) napsal(a): > Hi > > my mapping is > > "mongodocid":{"boost":1.0, > "store": "yes", > "type": "string"}, > "fulltext": {"boost": 1.0, > "index": "analyzed", > "store": "yes", > "type": "string", > "term\_vector" : "with\_positions\_offsets"}} > to highlight the complete fulltext i am setting number\_of\_framgments > to 0. > > if i do the following lucene like string query: > > '{"highlight": {"pre\_tags": "**", "fields": {"fulltext": > {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": > {"query\_string": {"query": "fulltext:test"}}, "size": 100}' > > for some documents in the resulset the lengh of the highlighted > fulltext is smaller than ten fulltext itself. Since i am setting > number\_of\_fragments to 0 and pre\_tags/post\_tags are added this should > not happen. > > Now comes the strange behaviour. if i only search for one of this > failing elements by doing this > > {"highlight": {"pre\_tags": "**", "fields": {"fulltext": > {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": > {"query\_string": {"query": "fulltext:test AND mongodocid: > 4d0a861c2ebef6032c00b1ec"}}, "size": 100} > > then all works fine. > > Any idea? ---
### Author: ![locojay](https://avatars.discourse-cdn.com/v4/letter/l/838e76/32.png) [@locojay](https://discuss.elastic.co/u/locojay) #### Post date: [December 22, 2010, 7:39pm UTC](https://discuss.elastic.co/t/highlight-problem/3692/3 "2010-12-22T19:39:12Z")
Hi i m using 0.13.1 On Dec 22, 2010, at 2:36 PM, Lukáš Vlček wrote: > Hi, which version of ES are you using? > Regards, > Lukas > > Dne 22.12.2010 20:29 "locojay" [jassin.meknassi@googlemail.com](mailto:jassin.meknassi@googlemail.com) napsal(a): > > > Hi > > > > my mapping is > > > > "mongodocid":{"boost":1.0, > > "store": "yes", > > "type": "string"}, > > "fulltext": {"boost": 1.0, > > "index": "analyzed", > > "store": "yes", > > "type": "string", > > "term\_vector" : "with\_positions\_offsets"}} > > to highlight the complete fulltext i am setting number\_of\_framgments > > to 0. > > > > if i do the following lucene like string query: > > > > '{"highlight": {"pre\_tags": "**", "fields": {"fulltext": > > {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": > > {"query\_string": {"query": "fulltext:test"}}, "size": 100}' > > > > for some documents in the resulset the lengh of the highlighted > > fulltext is smaller than ten fulltext itself. Since i am setting > > number\_of\_fragments to 0 and pre\_tags/post\_tags are added this should > > not happen. > > > > Now comes the strange behaviour. if i only search for one of this > > failing elements by doing this > > > > {"highlight": {"pre\_tags": "**", "fields": {"fulltext": > > {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": > > {"query\_string": {"query": "fulltext:test AND mongodocid: > > 4d0a861c2ebef6032c00b1ec"}}, "size": 100} > > > > then all works fine. > > > > Any idea? ---
### Author: ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1) #### Post date: [December 22, 2010, 7:41pm UTC](https://discuss.elastic.co/t/highlight-problem/3692/4 "2010-12-22T19:41:25Z")
Can you try master? There has been fixed the following bug > > > This is happening on version 0.12.1. When highlighting is specified, some of the… highlight fields come complete, and some are broken. When those documents are access directly by providing the direct index as "from" and setting "size" as 1, it comes fine: Reproduce code is below (bash script): > > \`\`\` > curl -XPUT 'http://server:9200/indexname/highbug/\_mapping' -d ' > { > "highbug": > { > "properties": > { > > "body": > { > "store": "yes", > "term\_vector": "with\_positions\_offsets", > "type": "string" > }, > "title": > { > "store": "yes", > "term\_vector": "with\_positions\_offsets", > "type": "string" > } > } > } > } > '; > > for (( i = 0; i \< 20; ++i )) > do > curl -XPUT "http://server:9200/indexname/highbug/$i" -d " > { > \\"body\\": \\"Test body\\", > \\"date\\": \\"$(date +%s)\\", > \\"title\\": \\"This is a test on the highlighting bug present in elasticsearch\\" > }"; > done > \`\`\` > > The call below returns the full highlighted fragment for the 1st and 5th results, and the 2nd through 4th results return "hting \bug\ present in elasticsearch" instead of "This is a test on the highlighting \bug\ present in elasticsearch". > > \`\`\` > curl -XGET 'http://server:9200/indexname/highbug/\_search?pretty=true' -d ' > {"query":{"query\_string":{"query":"bug"}},"from":0,"size":5,"highlight":{"fields":{"title":{"number\_of\_fragments":0}}}} > '; > \`\`\` Lukáš Dne 22.12.2010 20:39 "Jassin Meknassi" [jassin.meknassi@googlemail.com](mailto:jassin.meknassi@googlemail.com) napsal(a): > Hi i m using 0.13.1 > > On Dec 22, 2010, at 2:36 PM, Lukáš Vlček wrote: > > > Hi, which version of ES are you using? > > Regards, > > Lukas > > > > Dne 22.12.2010 20:29 "locojay" [jassin.meknassi@googlemail.com](mailto:jassin.meknassi@googlemail.com) > > napsal(a): > > > > > Hi > > > > > > my mapping is > > > > > > "mongodocid":{"boost":1.0, > > > "store": "yes", > > > "type": "string"}, > > > "fulltext": {"boost": 1.0, > > > "index": "analyzed", > > > "store": "yes", > > > "type": "string", > > > "term\_vector" : "with\_positions\_offsets"}} > > > to highlight the complete fulltext i am setting number\_of\_framgments > > > to 0. > > > > > > if i do the following lucene like string query: > > > > > > '{"highlight": {"pre\_tags": "**", "fields": {"fulltext": > > > {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": > > > {"query\_string": {"query": "fulltext:test"}}, "size": 100}' > > > > > > for some documents in the resulset the lengh of the highlighted > > > fulltext is smaller than ten fulltext itself. Since i am setting > > > number\_of\_fragments to 0 and pre\_tags/post\_tags are added this should > > > not happen. > > > > > > Now comes the strange behaviour. if i only search for one of this > > > failing elements by doing this > > > > > > {"highlight": {"pre\_tags": "**", "fields": {"fulltext": > > > {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": > > > {"query\_string": {"query": "fulltext:test AND mongodocid: > > > 4d0a861c2ebef6032c00b1ec"}}, "size": 100} > > > > > > then all works fine. > > > > > > Any idea? ---
### Author: ![Jassin\_MEKNASSI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jassin_meknassi/32/2982_2.png) [@Jassin\_MEKNASSI](https://discuss.elastic.co/u/Jassin_MEKNASSI) #### Post date: [December 22, 2010, 8:44pm UTC](https://discuss.elastic.co/t/highlight-problem/3692/5 "2010-12-22T20:44:41Z")
took longer to build the master. elasticsearch-0.14.0-SNAPSHOT instead of having a smaller size most have no entry: result['hits'][i]['highlight']['fulltext'] is an empty list On Dec 22, 2010, at 2:41 PM, Lukáš Vlček wrote: > Can you try master? There has been fixed the following bug [Highlighting bug where fragments are broken when number\_of\_fragments is 0 · Issue #479 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/issue/479) > > Lukáš > > Dne 22.12.2010 20:39 "Jassin Meknassi" [jassin.meknassi@googlemail.com](mailto:jassin.meknassi@googlemail.com) napsal(a): > > > Hi i m using 0.13.1 > > > > On Dec 22, 2010, at 2:36 PM, Lukáš Vlček wrote: > > > > > Hi, which version of ES are you using? > > > Regards, > > > Lukas > > > > > > Dne 22.12.2010 20:29 "locojay" [jassin.meknassi@googlemail.com](mailto:jassin.meknassi@googlemail.com) napsal(a): > > > > > > > Hi > > > > > > > > my mapping is > > > > > > > > "mongodocid":{"boost":1.0, > > > > "store": "yes", > > > > "type": "string"}, > > > > "fulltext": {"boost": 1.0, > > > > "index": "analyzed", > > > > "store": "yes", > > > > "type": "string", > > > > "term\_vector" : "with\_positions\_offsets"}} > > > > to highlight the complete fulltext i am setting number\_of\_framgments > > > > to 0. > > > > > > > > if i do the following lucene like string query: > > > > > > > > '{"highlight": {"pre\_tags": "**", "fields": {"fulltext": > > > > {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": > > > > {"query\_string": {"query": "fulltext:test"}}, "size": 100}' > > > > > > > > for some documents in the resulset the lengh of the highlighted > > > > fulltext is smaller than ten fulltext itself. Since i am setting > > > > number\_of\_fragments to 0 and pre\_tags/post\_tags are added this should > > > > not happen. > > > > > > > > Now comes the strange behaviour. if i only search for one of this > > > > failing elements by doing this > > > > > > > > {"highlight": {"pre\_tags": "**", "fields": {"fulltext": > > > > {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": > > > > {"query\_string": {"query": "fulltext:test AND mongodocid: > > > > 4d0a861c2ebef6032c00b1ec"}}, "size": 100} > > > > > > > > then all works fine. > > > > > > > > Any idea? ---
### Author: ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1) #### Post date: [December 22, 2010, 8:49pm UTC](https://discuss.elastic.co/t/highlight-problem/3692/6 "2010-12-22T20:49:37Z")
Can you please gist some curl recreation? On Wed, Dec 22, 2010 at 9:44 PM, Jassin MEKNASSI [jassin.meknassi@gmail.com](mailto:jassin.meknassi@gmail.com)wrote: > took longer to build the master. elasticsearch-0.14.0-SNAPSHOT > > instead of having a smaller size most have no entry: > > result['hits'][i]['highlight']['fulltext'] is an empty list > > On Dec 22, 2010, at 2:41 PM, Lukáš Vlček wrote: > > Can you try master? There has been fixed the following bug > [Highlighting bug where fragments are broken when number\_of\_fragments is 0 · Issue #479 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/issue/479) > > Lukáš > Dne 22.12.2010 20:39 "Jassin Meknassi" [jassin.meknassi@googlemail.com](mailto:jassin.meknassi@googlemail.com) > napsal(a): > > > Hi i m using 0.13.1 > > > > On Dec 22, 2010, at 2:36 PM, Lukáš Vlček wrote: > > > > > Hi, which version of ES are you using? > > > Regards, > > > Lukas > > > > > > Dne 22.12.2010 20:29 "locojay" [jassin.meknassi@googlemail.com](mailto:jassin.meknassi@googlemail.com) > > > napsal(a): > > > > > > > Hi > > > > > > > > my mapping is > > > > > > > > "mongodocid":{"boost":1.0, > > > > "store": "yes", > > > > "type": "string"}, > > > > "fulltext": {"boost": 1.0, > > > > "index": "analyzed", > > > > "store": "yes", > > > > "type": "string", > > > > "term\_vector" : "with\_positions\_offsets"}} > > > > to highlight the complete fulltext i am setting number\_of\_framgments > > > > to 0. > > > > > > > > if i do the following lucene like string query: > > > > > > > > '{"highlight": {"pre\_tags": "**", "fields": {"fulltext": > > > > {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": > > > > {"query\_string": {"query": "fulltext:test"}}, "size": 100}' > > > > > > > > for some documents in the resulset the lengh of the highlighted > > > > fulltext is smaller than ten fulltext itself. Since i am setting > > > > number\_of\_fragments to 0 and pre\_tags/post\_tags are added this should > > > > not happen. > > > > > > > > Now comes the strange behaviour. if i only search for one of this > > > > failing elements by doing this > > > > > > > > {"highlight": {"pre\_tags": "**", "fields": {"fulltext": > > > > {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": > > > > {"query\_string": {"query": "fulltext:test AND mongodocid: > > > > 4d0a861c2ebef6032c00b1ec"}}, "size": 100} > > > > > > > > then all works fine. > > > > > > > > Any idea? ---
### Author: ![Jassin\_MEKNASSI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jassin_meknassi/32/2982_2.png) [@Jassin\_MEKNASSI](https://discuss.elastic.co/u/Jassin_MEKNASSI) #### Post date: [December 23, 2010, 12:10am UTC](https://discuss.elastic.co/t/highlight-problem/3692/7 "2010-12-23T00:10:21Z")
unfortunately its confidential data. reindexing on the failing docs using master works. I will rebuild my index overnight and see if it works Thanks for the help On Dec 22, 2010, at 3:49 PM, Lukáš Vlček wrote: > Can you please gist some curl recreation? > > On Wed, Dec 22, 2010 at 9:44 PM, Jassin MEKNASSI [jassin.meknassi@gmail.com](mailto:jassin.meknassi@gmail.com) wrote: > took longer to build the master. elasticsearch-0.14.0-SNAPSHOT > > instead of having a smaller size most have no entry: > > result['hits'][i]['highlight']['fulltext'] is an empty list > > On Dec 22, 2010, at 2:41 PM, Lukáš Vlček wrote: > > > Can you try master? There has been fixed the following bug [Highlighting bug where fragments are broken when number\_of\_fragments is 0 · Issue #479 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/issue/479) > > > > Lukáš > > > > Dne 22.12.2010 20:39 "Jassin Meknassi" [jassin.meknassi@googlemail.com](mailto:jassin.meknassi@googlemail.com) napsal(a): > > > > > Hi i m using 0.13.1 > > > > > > On Dec 22, 2010, at 2:36 PM, Lukáš Vlček wrote: > > > > > > > Hi, which version of ES are you using? > > > > Regards, > > > > Lukas > > > > > > > > Dne 22.12.2010 20:29 "locojay" [jassin.meknassi@googlemail.com](mailto:jassin.meknassi@googlemail.com) napsal(a): > > > > > > > > > Hi > > > > > > > > > > my mapping is > > > > > > > > > > "mongodocid":{"boost":1.0, > > > > > "store": "yes", > > > > > "type": "string"}, > > > > > "fulltext": {"boost": 1.0, > > > > > "index": "analyzed", > > > > > "store": "yes", > > > > > "type": "string", > > > > > "term\_vector" : "with\_positions\_offsets"}} > > > > > to highlight the complete fulltext i am setting number\_of\_framgments > > > > > to 0. > > > > > > > > > > if i do the following lucene like string query: > > > > > > > > > > '{"highlight": {"pre\_tags": "**", "fields": {"fulltext": > > > > > {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": > > > > > {"query\_string": {"query": "fulltext:test"}}, "size": 100}' > > > > > > > > > > for some documents in the resulset the lengh of the highlighted > > > > > fulltext is smaller than ten fulltext itself. Since i am setting > > > > > number\_of\_fragments to 0 and pre\_tags/post\_tags are added this should > > > > > not happen. > > > > > > > > > > Now comes the strange behaviour. if i only search for one of this > > > > > failing elements by doing this > > > > > > > > > > {"highlight": {"pre\_tags": "**", "fields": {"fulltext": > > > > > {"number\_of\_fragments": 0}}, "post\_tags": "**"}, "query": > > > > > {"query\_string": {"query": "fulltext:test AND mongodocid: > > > > > 4d0a861c2ebef6032c00b1ec"}}, "size": 100} > > > > > > > > > > then all works fine. > > > > > > > > > > Any idea? ---
### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system) #### Post date: [July 6, 2017, 4:14am UTC](https://discuss.elastic.co/t/highlight-problem/3692/8 "2017-07-06T04:14:56Z")