# Different behaviour b/w custom and original Word Delimiter Token Filter **URL:** \ **Category:** Elasticsearch\ **Created:** [February 4, 2013, 5:20am UTC](https://discuss.elastic.co/t/different-behaviour-b-w-custom-and-original-word-delimiter-token-filter/10613 "2013-02-04T05:20:48Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Varun\_Thacker](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/varun_thacker/32/2472_2.png) [@Varun\_Thacker](https://discuss.elastic.co/u/Varun_Thacker)\ **Post date:** [February 4, 2013, 5:20am UTC](https://discuss.elastic.co/t/different-behaviour-b-w-custom-and-original-word-delimiter-token-filter/10613/1 "2013-02-04T05:20:48Z")
So I have 2 mappings one where I use the default Word Delimiter Token Filter and one where I use a custom one but don't really change any of the defaults. Both behave different behaviour. I'm not sure why this is happening. I've pasted the mappings here : [https://gist.github.com/4705144](https://gist.github.com/4705144) My aim to use a custom Word Delimiter Token Filter is to change "stem\_english\_possessive" to false. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\ **Post date:** [February 4, 2013, 9:37am UTC](https://discuss.elastic.co/t/different-behaviour-b-w-custom-and-original-word-delimiter-token-filter/10613/2 "2013-02-04T09:37:40Z")
Hi Varun, I don't see different behaviour between the two analyzer requests with version 0.20.4 What version are you using? Martijn On 4 February 2013 06:20, Varun Thacker [varunthacker1989@gmail.com](mailto:varunthacker1989@gmail.com) wrote: > So I have 2 mappings one where I use the default Word Delimiter Token > Filter and one where I use a custom one but don't really change any of the > defaults. Both behave different behaviour. I'm not sure why this is > happening. > > I've pasted the mappings here : [Word Delimiter Token Filter · GitHub](https://gist.github.com/4705144) > > My aim to use a custom Word Delimiter Token Filter is to change > "stem\_english\_possessive" to false. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Met vriendelijke groet, Martijn van Groningen -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Varun\_Thacker](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/varun_thacker/32/2472_2.png) [@Varun\_Thacker](https://discuss.elastic.co/u/Varun_Thacker)\ **Post date:** [February 4, 2013, 11:12am UTC](https://discuss.elastic.co/t/different-behaviour-b-w-custom-and-original-word-delimiter-token-filter/10613/3 "2013-02-04T11:12:41Z")
Hi Martijn, I'm using a build against master with all the latest commits. I've posted the results which I am seeing in the comments section ( [Word Delimiter Token Filter · GitHub](https://gist.github.com/4705144)) On Mon, Feb 4, 2013 at 3:07 PM, Martijn v Groningen \< [martijn.v.groningen@gmail.com](mailto:martijn.v.groningen@gmail.com)\> wrote: > Hi Varun, > > I don't see different behaviour between the two analyzer requests with > version 0.20.4 > What version are you using? > > Martijn > > On 4 February 2013 06:20, Varun Thacker [varunthacker1989@gmail.com](mailto:varunthacker1989@gmail.com)wrote: > > > So I have 2 mappings one where I use the default Word Delimiter Token > > Filter and one where I use a custom one but don't really change any of the > > defaults. Both behave different behaviour. I'm not sure why this is > > happening. > > > > I've pasted the mappings here : [Word Delimiter Token Filter · GitHub](https://gist.github.com/4705144) > > > > My aim to use a custom Word Delimiter Token Filter is to change > > "stem\_english\_possessive" to false. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > Met vriendelijke groet, > > Martijn van Groningen > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Regards, Varun Thacker [http://www.vthacker.in/](http://www.vthacker.in/) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [February 5, 2013, 12:25pm UTC](https://discuss.elastic.co/t/different-behaviour-b-w-custom-and-original-word-delimiter-token-filter/10613/4 "2013-02-05T12:25:00Z")
That looks like a bug that was introduced during Lucene 4 migration. We will fix it. On Monday, February 4, 2013 6:12:41 AM UTC-5, Varun Thacker wrote: > Hi Martijn, > > I'm using a build against master with all the latest commits. > > I've posted the results which I am seeing in the comments section ( > [Word Delimiter Token Filter · GitHub](https://gist.github.com/4705144)) > > On Mon, Feb 4, 2013 at 3:07 PM, Martijn v Groningen \< > [martijn.v...@gmail.com](mailto:martijn.v...@gmail.com) \\> wrote: > > > Hi Varun, > > > > I don't see different behaviour between the two analyzer requests with > > version 0.20.4 > > What version are you using? > > > > Martijn > > > > On 4 February 2013 06:20, Varun Thacker \<[varuntha...@gmail.com](mailto:varuntha...@gmail.com)\ > > > > > wrote: > > > > > So I have 2 mappings one where I use the default Word Delimiter Token > > > Filter and one where I use a custom one but don't really change any of the > > > defaults. Both behave different behaviour. I'm not sure why this is > > > happening. > > > > > > I've pasted the mappings here : [Word Delimiter Token Filter · GitHub](https://gist.github.com/4705144) > > > > > > My aim to use a custom Word Delimiter Token Filter is to change > > > "stem\_english\_possessive" to false. > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > Met vriendelijke groet, > > > > Martijn van Groningen > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > > Regards, > Varun Thacker > [http://www.vthacker.in/](http://www.vthacker.in/) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [February 5, 2013, 12:59pm UTC](https://discuss.elastic.co/t/different-behaviour-b-w-custom-and-original-word-delimiter-token-filter/10613/5 "2013-02-05T12:59:55Z")
I forgot to mention that \u0027 is json notation and when you use it in the \_analyze request body it's not getting converted to an apostrophe. You can use ''' or quotes instead: curl -XGET 'localhost:9200/test2/\_analyze?analyzer=myAnalyzer&pretty=true' -d 'world-class player wouldn'''t' curl -XGET 'localhost:9200/test2/\_analyze?analyzer=myAnalyzer&pretty=true' -d "world-class player wouldn't" On Tuesday, February 5, 2013 7:25:00 AM UTC-5, Igor Motov wrote: > That looks like a bug that was introduced during Lucene 4 migration. We > will fix it. > > On Monday, February 4, 2013 6:12:41 AM UTC-5, Varun Thacker wrote: > > > Hi Martijn, > > > > I'm using a build against master with all the latest commits. > > > > I've posted the results which I am seeing in the comments section ( > > [Word Delimiter Token Filter · GitHub](https://gist.github.com/4705144)) > > > > On Mon, Feb 4, 2013 at 3:07 PM, Martijn v Groningen \< > > [martijn.v...@gmail.com](mailto:martijn.v...@gmail.com)\> wrote: > > > > > Hi Varun, > > > > > > I don't see different behaviour between the two analyzer requests with > > > version 0.20.4 > > > What version are you using? > > > > > > Martijn > > > > > > On 4 February 2013 06:20, Varun Thacker [varuntha...@gmail.com](mailto:varuntha...@gmail.com) wrote: > > > > > > > So I have 2 mappings one where I use the default Word Delimiter Token > > > > Filter and one where I use a custom one but don't really change any of the > > > > defaults. Both behave different behaviour. I'm not sure why this is > > > > happening. > > > > > > > > I've pasted the mappings here : [Word Delimiter Token Filter · GitHub](https://gist.github.com/4705144) > > > > > > > > My aim to use a custom Word Delimiter Token Filter is to change > > > > "stem\_english\_possessive" to false. > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > Met vriendelijke groet, > > > > > > Martijn van Groningen > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > > > Regards, > > Varun Thacker > > [http://www.vthacker.in/](http://www.vthacker.in/) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:52am UTC](https://discuss.elastic.co/t/different-behaviour-b-w-custom-and-original-word-delimiter-token-filter/10613/6 "2017-07-06T02:52:59Z")