# Handling hyphenated words like "e-mail" with tokenizer/stemmer **URL:** \ **Category:** Elasticsearch\ **Created:** [November 16, 2016, 7:24pm UTC](https://discuss.elastic.co/t/handling-hyphenated-words-like-e-mail-with-tokenizer-stemmer/66295 "2016-11-16T19:24:04Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![David\_Steiner](https://avatars.discourse-cdn.com/v4/letter/d/90ced4/32.png) [@David\_Steiner](https://discuss.elastic.co/u/David_Steiner)\ **Post date:** [November 16, 2016, 7:24pm UTC](https://discuss.elastic.co/t/handling-hyphenated-words-like-e-mail-with-tokenizer-stemmer/66295/1 "2016-11-16T19:24:04Z")
Ideally, I would like to be able to stem "e-mail" to "email" so that whether someone has typed it one way or the other, a search in the stemmed field will find it. Currently, it appears the standard tokenizer changes "e-mail" into "e" and "mail", which isn't good because now I can't stem "e-mail" and now "mail" and "e-mail" are combined. I can't use "whitespace" as a tokenizer because if the text I'm indexing contains other punctuation, such as "He said, 'send me an e-mail!'", "send" isn't even indexed correctly - it's indexed as "'send". So, I guess I'd like to be able to tokenize in a standard way except that I want hyphens retained so that I have a chance to stem hyphenated words. Is there a way to do this? I hoping that if it involves the "pattern" tokenizer that there's already a known pattern for this? ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [November 16, 2016, 9:08pm UTC](https://discuss.elastic.co/t/handling-hyphenated-words-like-e-mail-with-tokenizer-stemmer/66295/2 "2016-11-16T21:08:35Z")
I wrote a hyphen tokenizer for this, for example [https://github.com/jprante/elasticsearch-plugin-bundle/blob/master/src/test/java/org/xbib/elasticsearch/index/analysis/hyphen/HyphenTokenizerTests.java](https://github.com/jprante/elasticsearch-plugin-bundle/blob/master/src/test/java/org/xbib/elasticsearch/index/analysis/hyphen/HyphenTokenizerTests.java) The tokenizer is bundled with all my other analyzers/tokenizers/token filters in a plugin, see > **[jprante/elasticsearch-plugin-bundle](https://github.com/jprante/elasticsearch-plugin-bundle)** > > elasticsearch-plugin-bundle - A bundle of useful Elasticsearch plugins ---
**Author:** ![David\_Steiner](https://avatars.discourse-cdn.com/v4/letter/d/90ced4/32.png) [@David\_Steiner](https://discuss.elastic.co/u/David_Steiner)\ **Post date:** [November 16, 2016, 9:32pm UTC](https://discuss.elastic.co/t/handling-hyphenated-words-like-e-mail-with-tokenizer-stemmer/66295/3 "2016-11-16T21:32:42Z")
I'm using elasticsearch 5.0 (and I'm new to elasticsearch). Will this work with 5.0? ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [November 16, 2016, 9:34pm UTC](https://discuss.elastic.co/t/handling-hyphenated-words-like-e-mail-with-tokenizer-stemmer/66295/4 "2016-11-16T21:34:09Z")
I see. I need to update my project. ---
**Author:** ![David\_Steiner](https://avatars.discourse-cdn.com/v4/letter/d/90ced4/32.png) [@David\_Steiner](https://discuss.elastic.co/u/David_Steiner)\ **Post date:** [November 29, 2016, 8:52pm UTC](https://discuss.elastic.co/t/handling-hyphenated-words-like-e-mail-with-tokenizer-stemmer/66295/5 "2016-11-29T20:52:09Z")
I'm assuming that you're reply means that what you have here will not work with 5.0 as it is? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 27, 2016, 8:53pm UTC](https://discuss.elastic.co/t/handling-hyphenated-words-like-e-mail-with-tokenizer-stemmer/66295/6 "2016-12-27T20:53:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.