# Using PatternTokenizer **URL:** \ **Category:** Elasticsearch\ **Created:** [July 25, 2010, 9:28am UTC](https://discuss.elastic.co/t/using-patterntokenizer/3148 "2010-07-25T09:28:25Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\ **Post date:** [July 25, 2010, 9:28am UTC](https://discuss.elastic.co/t/using-patterntokenizer/3148/1 "2010-07-25T09:28:25Z")
Hello, Is it correct that in order to use the PatternTokenizer, one would need to implement a plugin similar to icu? Thanks, Paul ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 25, 2010, 6:50pm UTC](https://discuss.elastic.co/t/using-patterntokenizer/3148/2 "2010-07-25T18:50:38Z")
Yes, but it can be part of the built in analyzers in elasticsearch (I assume you refer to the one in Lucene). -shay.banon On Sun, Jul 25, 2010 at 12:28 PM, Paul [ppearcy@gmail.com](mailto:ppearcy@gmail.com) wrote: > Hello, > Is it correct that in order to use the PatternTokenizer, one would > need to implement a plugin similar to icu? > > Thanks, > Paul ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 25, 2010, 7:42pm UTC](https://discuss.elastic.co/t/using-patterntokenizer/3148/3 "2010-07-25T19:42:55Z")
Add this: [Analysis: Add pattern analyzer · Issue #276 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/issue/276). On Sun, Jul 25, 2010 at 9:50 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > Yes, but it can be part of the built in analyzers in elasticsearch (I > assume you refer to the one in Lucene). > > -shay.banon > > On Sun, Jul 25, 2010 at 12:28 PM, Paul [ppearcy@gmail.com](mailto:ppearcy@gmail.com) wrote: > > > Hello, > > Is it correct that in order to use the PatternTokenizer, one would > > need to implement a plugin similar to icu? > > > > Thanks, > > Paul ---
**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\ **Post date:** [July 25, 2010, 11:16pm UTC](https://discuss.elastic.co/t/using-patterntokenizer/3148/4 "2010-07-25T23:16:23Z")
Yeah, it probably makes sense to have it built in. I'd be happy to create a fork and submit it. Would plan on exposing the pattern, lowercase, and stopwords options that map directly to Lucene's PatternAnalyzer inputs. A separate pattern tokenizer would be nice to combine with other options, but that doesn't appear to exist in Lucene (though Solr has a more flexible version based on regex grouping that will probably be available with the Lucene/Solr merge). Not that it would be hard to write, just don't need it for my use case. Thanks, Paul On Jul 25, 12:50 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > Yes, but it can be part of the built in analyzers in elasticsearch (I assume > you refer to the one in Lucene). > > -shay.banon > > On Sun, Jul 25, 2010 at 12:28 PM, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > Hello, > > Is it correct that in order to use the PatternTokenizer, one would > > need to implement a plugin similar to icu? > > > Thanks, > > Paul ---
**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\ **Post date:** [July 26, 2010, 1:55am UTC](https://discuss.elastic.co/t/using-patterntokenizer/3148/5 "2010-07-26T01:55:47Z")
Huh, somehow the Nabble (which shows your response referencing [Analysis: Add pattern analyzer · Issue #276 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/issue/276)) and google groups which doesn't are out of sync. Anyway, thanks a ton! Seems straight forward and I'll let you know if there are any issues. Best Regards, Paul On Jul 25, 5:16 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > Yeah, it probably makes sense to have it built in. I'd be happy to > create a fork and submit it. Would plan on exposing the pattern, > lowercase, and stopwords options that map directly to Lucene's > PatternAnalyzer inputs. > > A separate pattern tokenizer would be nice to combine with other > options, but that doesn't appear to exist in Lucene (though Solr has a > more flexible version based on regex grouping that will probably be > available with the Lucene/Solr merge). Not that it would be hard to > write, just don't need it for my use case. > > Thanks, > Paul > > On Jul 25, 12:50 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > Yes, but it can be part of the built in analyzers in elasticsearch (I assume > > you refer to the one in Lucene). > > > -shay.banon > > > On Sun, Jul 25, 2010 at 12:28 PM, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > > > Hello, > > > Is it correct that in order to use the PatternTokenizer, one would > > > need to implement a plugin similar to icu? > > > > Thanks, > > > Paul ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:21am UTC](https://discuss.elastic.co/t/using-patterntokenizer/3148/6 "2017-07-06T04:21:37Z")