# UAX\_URL\_EMAIL tokenizer problem

**URL:** <https://discuss.elastic.co/t/uax-url-email-tokenizer-problem/72230>\
**Category:** Elasticsearch\
**Created:** [January 19, 2017, 9:47pm UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer-problem/72230 "2017-01-19T21:47:49Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![pplum](https://avatars.discourse-cdn.com/v4/letter/p/f19dbf/32.png) [@pplum](https://discuss.elastic.co/u/pplum)\
**Post date:** [January 19, 2017, 9:47pm UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer-problem/72230/1 "2017-01-19T21:47:49Z")

</div>

Hey guys,

does uax\_url\_email tokenizer not work with short urls like "[google24.com](http://google24.com)"? Was wondering if you guys were getting the same results and also what recommendations you guys have for workarounds

Goal: trying to tokenize "[google24.com](http://google24.com)"  
Problem: Trying to correctly tokenize: "[google24.com](http://google24.com)"  
Result: I get two tokens: "google24" and "com"

input (fail):

```
curl -XPOST 'localhost:9200/_analyze?pretty' -d'
{
    "tokenizer": "uax_url_email",
    "text": "google24.com"
}'

```

output:

```
{
  "tokens" : [ {
    "token" : "google24",
    "start_offset" : 0,
    "end_offset" : 8,
    "type" : "<ALPHANUM>",
    "position" : 0
  }, {
    "token" : "com",
    "start_offset" : 9,
    "end_offset" : 12,
    "type" : "<ALPHANUM>",
    "position" : 1
  } ]
}

```

* * *

input (pass):

```
curl -XPOST 'localhost:9200/_analyze?pretty' -d'
{
    "tokenizer": "uax_url_email",
    "text": "google.com"
}'

```

output, notice that the input is detected as alphanum and not url

```
{
  "tokens" : [ {
    "token" : "google.com",
    "start_offset" : 0,
    "end_offset" : 10,
    "type" : "<ALPHANUM>",
    "position" : 0
  } ]
}

```

* * *

input (pass), notice the space character after the ".com"

```
curl -XPOST localhost:9200/_analyze?pretty' -d'
{
  "tokenizer": "uax_url_email",
  "text": "google24.com "
}'

```

output

```
{
  "tokens" : [ {
    "token" : "google24.com",
    "start_offset" : 0,
    "end_offset" : 12,
    "type" : "<URL>",
    "position" : 0
  } ]
}

```

Questions:

1. why does adding a space after the url in the 3rd example produce the desired results?
2. why does it appear that urls with numeric characters before the ".com" arent being tokenized correctly

Thank you everyone for you help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 16, 2017, 9:47pm UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer-problem/72230/2 "2017-02-16T21:47:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.