# Inconsistent number of tokens with uax\_url\_email

**URL:** <https://discuss.elastic.co/t/inconsistent-number-of-tokens-with-uax-url-email/47327>\
**Category:** Elasticsearch\
**Created:** [April 13, 2016, 11:35pm UTC](https://discuss.elastic.co/t/inconsistent-number-of-tokens-with-uax-url-email/47327 "2016-04-13T23:35:56Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![joey.solis](https://avatars.discourse-cdn.com/v4/letter/j/34f0e0/32.png) [@joey.solis](https://discuss.elastic.co/u/joey.solis)\
**Post date:** [April 13, 2016, 11:35pm UTC](https://discuss.elastic.co/t/inconsistent-number-of-tokens-with-uax-url-email/47327/1 "2016-04-13T23:35:56Z")

</div>

I would like the first tokenizing below (#1) to result in 1 token. The only difference to #2 is that the domain starts with a number. Any feedback is greatly appreciated. Thank you.

#1. Results in 2 tokens:  
curl -XGET '[http://localhost:9200/\_analyze?tokenizer=uax\_url\_email&text=www.123abc.com](http://localhost:9200/_analyze?tokenizer=uax_url_email&text=www.123abc.com)'

{"tokens":[{"token":"www","start\_offset":0,"end\_offset":3,"type":"","position":0},{"token":"[123abc.com](http://123abc.com)","start\_offset":4,"end\_offset":14,"type":"","position":1}]}

#2. Results in 1 token:  
curl -XGET '[http://localhost:9200/\_analyze?tokenizer=uax\_url\_email&text=www.a123bc.com](http://localhost:9200/_analyze?tokenizer=uax_url_email&text=www.a123bc.com)'  
{"tokens":[{"token":"[www.a123bc.com](http://www.a123bc.com)","start\_offset":0,"end\_offset":14,"type":"","position":0}]}

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [April 14, 2016, 1:23pm UTC](https://discuss.elastic.co/t/inconsistent-number-of-tokens-with-uax-url-email/47327/2 "2016-04-14T13:23:41Z")

</div>

Hey,

The way I understand this you need to provide a real URL which contains a scheme like `http://` in order to trigger make use of the tokenizer. You can easily reproduce this by not specifying the tokenizer and getting the exact same results. Those differ when using a real URL.

--Alex

---

<div class="post-metadata">

**Author:** ![joey.solis](https://avatars.discourse-cdn.com/v4/letter/j/34f0e0/32.png) [@joey.solis](https://discuss.elastic.co/u/joey.solis)\
**Post date:** [April 14, 2016, 2:53pm UTC](https://discuss.elastic.co/t/inconsistent-number-of-tokens-with-uax-url-email/47327/3 "2016-04-14T14:53:03Z")

</div>

Hi Alex,

Thank you for your answer. That helps us understand the tokenizer and find a solution since out data/urls do not have schemes.

Joey

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:59pm UTC](https://discuss.elastic.co/t/inconsistent-number-of-tokens-with-uax-url-email/47327/4 "2017-07-05T22:59:10Z")

</div>