# Highlight output changed in 0.16 **URL:** \ **Category:** Elasticsearch\ **Created:** [April 28, 2011, 6:54am UTC](https://discuss.elastic.co/t/highlight-output-changed-in-0-16/4306 "2011-04-28T06:54:51Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![ruflin\_2](https://avatars.discourse-cdn.com/v4/letter/r/a8b319/32.png) [@ruflin\_2](https://discuss.elastic.co/u/ruflin_2)\ **Post date:** [April 28, 2011, 6:54am UTC](https://discuss.elastic.co/t/highlight-output-changed-in-0-16/4306/1 "2011-04-28T06:54:51Z")
With 0.16 the highlight output changed a little bit: 1. Elastica\_Query\_HighlightTest::testHightlightSearch Failed asserting that two arrays are equal. --- Expected +++ Actual @@ @@ Array ( [email] =\> Array ( - ``` [0] => test@test.com ``` - ``` [0] => test@test.com ) ) The one with only 1 _was in version 0.15.2, the output with multiple is in 0.16. Is this change as expected?_ ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [April 28, 2011, 8:14am UTC](https://discuss.elastic.co/t/highlight-output-changed-in-0-16/4306/2 "2011-04-28T08:14:12Z")
Hi Ruflin > With 0.16 the highlight output changed a little bit: > - > > ``` > [0] => test@test.com > > ``` > > - > > ``` > [0] => test@ > ``` > > class="highlight"\>test.com > The one with only 1 _was in version 0.15.2, the output with > multiple is in 0.16. Is this change as expected?_ _I think it isn't the highlighting that has changed, but the default analyzer which now breaks up email addresses into several terms. Before, email addresses produced a single term._ _clint_ ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 28, 2011, 11:37am UTC](https://discuss.elastic.co/t/highlight-output-changed-in-0-16/4306/3 "2011-04-28T11:37:31Z")
Yes, thats the new behavior in Lucene 3.1. You can now specify a Lucene version on tokenizer/analyzer/... to revert to the old behavior. On Thursday, April 28, 2011 at 11:14 AM, Clinton Gormley wrote: > Hi Ruflin > > > With 0.16 the highlight output changed a little bit: > > > - [0] =\> _[test@test.com](mailto:test@test.com)_ > > > > - [0] =\> _test_@_[test.com](http://test.com)_ > > > The one with only 1 _was in version 0.15.2, the output with > > multiple is in 0.16. Is this change as expected?_ > > > > _I think it isn't the highlighting that has changed, but the default > analyzer which now breaks up email addresses into several terms. Before, > email addresses produced a single term._ > > > > _clint_ ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [April 28, 2011, 12:25pm UTC](https://discuss.elastic.co/t/highlight-output-changed-in-0-16/4306/4 "2011-04-28T12:25:55Z")
Hi, I did not have chance to get myself fully familiar with all new Lucene 3.1 analyzers but as far as I understand it is possible to create token filters specifically for emails, urls and paths based on uax\_url\_email tokenizer. Is this directly exposed in ES 0.16 ? Looking at the original rufin's code that would be the best solution IMHO (as he is having email addresses in the text). See EmailFilter in TestUAX29URLEmailTokenizer.java ( [http://svn.apache.org/viewvc/lucene/dev/tags/lucene\_solr\_3\_1/lucene/src/test/org/apache/lucene/analysis/TestUAX29URLEmailTokenizer.java?view=markup](http://svn.apache.org/viewvc/lucene/dev/tags/lucene_solr_3_1/lucene/src/test/org/apache/lucene/analysis/TestUAX29URLEmailTokenizer.java?view=markup) ) Regards, Lukas On Thu, Apr 28, 2011 at 1:37 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > Yes, thats the new behavior in Lucene 3.1. You can now specify a Lucene > version on tokenizer/analyzer/... to revert to the old behavior. > > On Thursday, April 28, 2011 at 11:14 AM, Clinton Gormley wrote: > > Hi Ruflin > > With 0.16 the highlight output changed a little bit: > > - [0] =\> _[test@test.com](mailto:test@test.com)_ > > - [0] =\> _test_@_[test.com](http://test.com)_ > > The one with only 1 _was in version 0.15.2, the output with > multiple is in 0.16. Is this change as expected?_ > > > > _I think it isn't the highlighting that has changed, but the default > analyzer which now breaks up email addresses into several terms. Before, > email addresses produced a single term._ > > > > _clint_ ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [April 28, 2011, 12:38pm UTC](https://discuss.elastic.co/t/highlight-output-changed-in-0-16/4306/5 "2011-04-28T12:38:39Z")
Hi Lukas > I did not have chance to get myself fully familiar with all new Lucene > 3.1 analyzers but as far as I understand it is possible to create > token filters specifically for emails, urls and paths based on > uax\_url\_email tokenizer. Is this directly exposed in ES 0.16 ? Yes it is: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. clint ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [April 28, 2011, 12:41pm UTC](https://discuss.elastic.co/t/highlight-output-changed-in-0-16/4306/6 "2011-04-28T12:41:51Z")
Correct URL for Path Hierarchy: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > Yes it is: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/uaxurlemail-tokenizer.html) > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/pathheirarchy-tokenizer.html) > > clint ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [April 28, 2011, 12:48pm UTC](https://discuss.elastic.co/t/highlight-output-changed-in-0-16/4306/7 "2011-04-28T12:48:31Z")
I was probably not clear, what I was asking about is if there is any option how to configure email filter. This means you have a text which contains several email addresses and the output would be only those email addresses. That is what that Lucene test does. On Thu, Apr 28, 2011 at 2:41 PM, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk)wrote: > Correct URL for Path Hierarchy: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/pathhierarchy-tokenizer.html) > > > Yes it is: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/uaxurlemail-tokenizer.html) > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/pathheirarchy-tokenizer.html) > > > clint ---
**Author:** ![ruflin\_2](https://avatars.discourse-cdn.com/v4/letter/r/a8b319/32.png) [@ruflin\_2](https://discuss.elastic.co/u/ruflin_2)\ **Post date:** [April 28, 2011, 1:07pm UTC](https://discuss.elastic.co/t/highlight-output-changed-in-0-16/4306/8 "2011-04-28T13:07:18Z")
@Clinton & Shay: Good to know. Then I will update the tests. On Apr 28, 1:37 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > Yes, thats the new behavior in Lucene 3.1. You can now specify a Lucene version on tokenizer/analyzer/... to revert to the old behavior. > > On Thursday, April 28, 2011 at 11:14 AM, Clinton Gormley wrote: > > > Hi Ruflin > > > > With 0.16 the highlight output changed a little bit: > > > > - [0] =\> _[t...@test.com](mailto:t...@test.com)_ > > > > > > - [0] =\> _test_@_[test.com](http://test.com)_ > > > > The one with only 1 _was in version 0.15.2, the output with > > > multiple is in 0.16. Is this change as expected?_ > > > > _> I think it isn't the highlighting that has changed, but the default > > analyzer which now breaks up email addresses into several terms. Before, > > email addresses produced a single term._ > > > > _> clint_ ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 28, 2011, 2:06pm UTC](https://discuss.elastic.co/t/highlight-output-changed-in-0-16/4306/9 "2011-04-28T14:06:39Z")
There isn't a built in EmailFilter, which is used in the test just to make sure the relevant token type is used. On Thursday, April 28, 2011 at 3:48 PM, Lukáš Vlček wrote: > I was probably not clear, what I was asking about is if there is any option how to configure email filter. This means you have a text which contains several email addresses and the output would be only those email addresses. That is what that Lucene test does. > > On Thu, Apr 28, 2011 at 2:41 PM, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk) wrote: > > > Correct URL for Path Hierarchy: > > > > [http://www.elasticsearch.org/guide/reference/index-modules/analysis/pathhierarchy-tokenizer.html](http://www.elasticsearch.org/guide/reference/index-modules/analysis/pathhierarchy-tokenizer.html) > > > > > Yes it is: > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/uaxurlemail-tokenizer.html) > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/pathheirarchy-tokenizer.html) > > > > > > clint ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [April 28, 2011, 2:52pm UTC](https://discuss.elastic.co/t/highlight-output-changed-in-0-16/4306/10 "2011-04-28T14:52:51Z")
Shay, I think those filters could be useful. For example if I am interested in emails or URLs and I am not interested in the stuffed text content inbetween. Just an idea for nice-to-have feature. Lukas On Thu, Apr 28, 2011 at 4:06 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > There isn't a built in EmailFilter, which is used in the test just to > make sure the relevant token type is used. > > On Thursday, April 28, 2011 at 3:48 PM, Lukáš Vlček wrote: > > I was probably not clear, what I was asking about is if there is any option > how to configure email filter. This means you have a text which contains > several email addresses and the output would be only those email addresses. > That is what that Lucene test does. > > On Thu, Apr 28, 2011 at 2:41 PM, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk)wrote: > > Correct URL for Path Hierarchy: > > [http://www.elasticsearch.org/guide/reference/index-modules/analysis/pathhierarchy-tokenizer.html](http://www.elasticsearch.org/guide/reference/index-modules/analysis/pathhierarchy-tokenizer.html) > > > Yes it is: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/uaxurlemail-tokenizer.html) > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/pathheirarchy-tokenizer.html) > > > clint ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 28, 2011, 8:37pm UTC](https://discuss.elastic.co/t/highlight-output-changed-in-0-16/4306/11 "2011-04-28T20:37:36Z")
Sounds good. On Thursday, April 28, 2011 at 5:52 PM, Lukáš Vlček wrote: > Shay, > > I think those filters could be useful. For example if I am interested in emails or URLs and I am not interested in the stuffed text content inbetween. Just an idea for nice-to-have feature. > > Lukas > > On Thu, Apr 28, 2011 at 4:06 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > There isn't a built in EmailFilter, which is used in the test just to make sure the relevant token type is used. > > On Thursday, April 28, 2011 at 3:48 PM, Lukáš Vlček wrote: > > > > > I was probably not clear, what I was asking about is if there is any option how to configure email filter. This means you have a text which contains several email addresses and the output would be only those email addresses. That is what that Lucene test does. > > > > > > On Thu, Apr 28, 2011 at 2:41 PM, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk) wrote: > > > > > > > Correct URL for Path Hierarchy: > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/pathhierarchy-tokenizer.html) > > > > > > > > > Yes it is: > > > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/uaxurlemail-tokenizer.html) > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/pathheirarchy-tokenizer.html) > > > > > > > > > > clint ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:07am UTC](https://discuss.elastic.co/t/highlight-output-changed-in-0-16/4306/12 "2017-07-06T04:07:26Z")