# ElasticSearch standard Analyzer - problematic case **URL:** \ **Category:** Elasticsearch\ **Created:** [April 3, 2014, 8:52am UTC](https://discuss.elastic.co/t/elasticsearch-standard-analyzer-problematic-case/16782 "2014-04-03T08:52:31Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Igor\_Romanov](https://avatars.discourse-cdn.com/v4/letter/i/e19adc/32.png) [@Igor\_Romanov](https://discuss.elastic.co/u/Igor_Romanov)\ **Post date:** [April 3, 2014, 8:52am UTC](https://discuss.elastic.co/t/elasticsearch-standard-analyzer-problematic-case/16782/1 "2014-04-03T08:52:31Z")
Hi I was analyzing some analyzer weird behaviour, and try to understand why it happens and how to fix it here what token I get for standard analyzer for text: "myemail@email.com:test1234" curl -XGET 'localhost:9200/\_analyze?analyzer=standard&pretty=true' -d 'myemail@email.com:test1234' { "tokens" : [ { "token" : "myemail", "start\_offset" : 0, "end\_offset" : 7, "type" : "", "position" : 1 }, { "token" : "[email.com](http://email.com):test1234", "start\_offset" : 8, "end\_offset" : 26, "type" : "", "position" : 2 } ] } so question is why I am getting that as one token: "[email.com](http://email.com):test1234" why it is not devided to tokens by . and : ? and what analyzer/tokenizer/filter can I use that can help with it? Thanks, Igor -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/826eb584-3408-404a-b87c-2c44e455bb65%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/826eb584-3408-404a-b87c-2c44e455bb65%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Robert\_Muir\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robert_muir_2/32/931_2.png) [@Robert\_Muir\_2](https://discuss.elastic.co/u/Robert_Muir_2)\ **Post date:** [April 3, 2014, 4:48pm UTC](https://discuss.elastic.co/t/elasticsearch-standard-analyzer-problematic-case/16782/2 "2014-04-03T16:48:30Z")
The standard analyzer doesn't really know anything about emails/URLs, its just implementing the Unicode tokenization rules. There is an extension of it that does know about these things (and tries to keep them as one token)... > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Maybe try this one and see if it works better for you? On Thu, Apr 3, 2014 at 4:52 AM, Igor Romanov [igorrom@gmail.com](mailto:igorrom@gmail.com) wrote: > Hi > > I was analyzing some analyzer weird behaviour, and try to understand why it > happens and how to fix it > > here what token I get for standard analyzer for text: > "[myemail@email.com](mailto:myemail@email.com):test1234" > > curl -XGET 'localhost:9200/\_analyze?analyzer=standard&pretty=true' -d > 'myemail@email.com:test1234' > { > "tokens" : [ { > "token" : "myemail", > "start\_offset" : 0, > "end\_offset" : 7, > "type" : "", > "position" : 1 > }, { > "token" : "[email.com](http://email.com):test1234", > "start\_offset" : 8, > "end\_offset" : 26, > "type" : "", > "position" : 2 > } ] > } > > so question is why I am getting that as one token: "[email.com](http://email.com):test1234" > > why it is not devided to tokens by . and : ? > > and what analyzer/tokenizer/filter can I use that can help with it? > > Thanks, > Igor > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/826eb584-3408-404a-b87c-2c44e455bb65%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/826eb584-3408-404a-b87c-2c44e455bb65%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAMUKNZWGsks9O5Y5qupAovgn6Vwa3EwVKju9WOeSmW3dQ-hPTA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMUKNZWGsks9O5Y5qupAovgn6Vwa3EwVKju9WOeSmW3dQ-hPTA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:38am UTC](https://discuss.elastic.co/t/elasticsearch-standard-analyzer-problematic-case/16782/3 "2017-07-06T01:38:31Z")