# Word\_delimiter with split\_on\_numerics removes all tokens

**URL:** <https://discuss.elastic.co/t/word-delimiter-with-split-on-numerics-removes-all-tokens/791>\
**Category:** Elasticsearch\
**Created:** [May 16, 2015, 10:22pm UTC](https://discuss.elastic.co/t/word-delimiter-with-split-on-numerics-removes-all-tokens/791 "2015-05-16T22:22:37Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![nacho](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nacho/32/44834_2.png) [@nacho](https://discuss.elastic.co/u/nacho)\
**Post date:** [May 16, 2015, 10:22pm UTC](https://discuss.elastic.co/t/word-delimiter-with-split-on-numerics-removes-all-tokens/791/1 "2015-05-16T22:22:37Z")

</div>

When analyzing `alpha 1a beta`, I want the outcome of tokens to be `[alpha 1 a beta]`. Why does `myAnalyzer` not do the trick?

```
POST myindex
{
  "settings" : {
    "analysis" : {
      "analyzer" : {
        "myAnalyzer" : {
          "type" : "custom",
          "tokenizer" : "standard",
          "filter" : ["split_on_numerics"]
        }
      },
      "filter" : {
        "split_on_numerics" : {
          "type" : "word_delimiter",
          "split_on_numerics" : true,
          "split_on_case_change" : false,
          "generate_word_parts" : false,
          "generate_number_parts" : false,
          "catenate_all" : false
        }
      }
    }
  }
}

```

Now when I run

```
GET /myindex/_analyze?analyzer=myAnalyzer&text=alpha 1a beta

```

no tokens are returned. Again, why?

---

<div class="post-metadata">

**Author:** ![Jason\_Wee](https://avatars.discourse-cdn.com/v4/letter/j/7ea924/32.png) [@Jason\_Wee](https://discuss.elastic.co/u/Jason_Wee)\
**Post date:** [May 17, 2015, 9:23am UTC](https://discuss.elastic.co/t/word-delimiter-with-split-on-numerics-removes-all-tokens/791/2 "2015-05-17T09:23:45Z")

</div>

```
curl -XPUT 'http://localhost:9200/myindex/?pretty' -d '
{
  "settings" : {
    "analysis" : {
      "analyzer" : {
        "myAnalyzer" : {
          "type" : "custom",
          "tokenizer" : "standard",
          "filter" : ["split_on_numerics"]
        }
      },
      "filter" : {
        "split_on_numerics" : {
          "type" : "word_delimiter",
          "split_on_numerics" : true,
          "split_on_case_change" : false,
          "generate_word_parts" : true,
          "generate_number_parts" : true,
          "catenate_all" : false
        }
      }
    }
  }
}'

curl -XGET 'localhost:9200/myindex/_analyze?pretty&analyzer=myAnalyzer' -d 'alpha 1a beta'
{
  "tokens" : [ {
    "token" : "alpha",
    "start_offset" : 0,
    "end_offset" : 5,
    "type" : "<ALPHANUM>",
    "position" : 1
  }, {
    "token" : "1",
    "start_offset" : 6,
    "end_offset" : 7,
    "type" : "<ALPHANUM>",
    "position" : 2
  }, {
    "token" : "a",
    "start_offset" : 7,
    "end_offset" : 8,
    "type" : "<ALPHANUM>",
    "position" : 3
  }, {
    "token" : "beta",
    "start_offset" : 9,
    "end_offset" : 13,
    "type" : "<ALPHANUM>",
    "position" : 4
  } ]
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:13am UTC](https://discuss.elastic.co/t/word-delimiter-with-split-on-numerics-removes-all-tokens/791/3 "2017-07-06T00:13:41Z")

</div>