Different behaviour b/w custom and original Word Delimiter Token Filter

Elastic Stack Elasticsearch
1

So I have 2 mappings one where I use the default Word Delimiter Token
Filter and one where I use a custom one but don't really change any of the
defaults. Both behave different behaviour. I'm not sure why this is
happening.

I've pasted the mappings here : https://gist.github.com/4705144

My aim to use a custom Word Delimiter Token Filter is to change
"stem_english_possessive" to false.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hi Varun,

I don't see different behaviour between the two analyzer requests with
version 0.20.4
What version are you using?

Martijn

On 4 February 2013 06:20, Varun Thacker varunthacker1989@gmail.com wrote:

So I have 2 mappings one where I use the default Word Delimiter Token
Filter and one where I use a custom one but don't really change any of the
defaults. Both behave different behaviour. I'm not sure why this is
happening.

I've pasted the mappings here : Word Delimiter Token Filter · GitHub

My aim to use a custom Word Delimiter Token Filter is to change
"stem_english_possessive" to false.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi Martijn,

I'm using a build against master with all the latest commits.

I've posted the results which I am seeing in the comments section (
Word Delimiter Token Filter · GitHub)

On Mon, Feb 4, 2013 at 3:07 PM, Martijn v Groningen <
martijn.v.groningen@gmail.com> wrote:

Hi Varun,

I don't see different behaviour between the two analyzer requests with
version 0.20.4
What version are you using?

Martijn

On 4 February 2013 06:20, Varun Thacker varunthacker1989@gmail.comwrote:

So I have 2 mappings one where I use the default Word Delimiter Token
Filter and one where I use a custom one but don't really change any of the
defaults. Both behave different behaviour. I'm not sure why this is
happening.

I've pasted the mappings here : Word Delimiter Token Filter · GitHub

My aim to use a custom Word Delimiter Token Filter is to change
"stem_english_possessive" to false.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Regards,
Varun Thacker
http://www.vthacker.in/

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

That looks like a bug that was introduced during Lucene 4 migration. We
will fix it.

On Monday, February 4, 2013 6:12:41 AM UTC-5, Varun Thacker wrote:

Hi Martijn,

I'm using a build against master with all the latest commits.

I've posted the results which I am seeing in the comments section (
Word Delimiter Token Filter · GitHub)

On Mon, Feb 4, 2013 at 3:07 PM, Martijn v Groningen <
martijn.v...@gmail.com <javascript:>> wrote:

Hi Varun,

I don't see different behaviour between the two analyzer requests with
version 0.20.4
What version are you using?

Martijn

On 4 February 2013 06:20, Varun Thacker <varuntha...@gmail.com<javascript:>

wrote:

So I have 2 mappings one where I use the default Word Delimiter Token
Filter and one where I use a custom one but don't really change any of the
defaults. Both behave different behaviour. I'm not sure why this is
happening.

I've pasted the mappings here : Word Delimiter Token Filter · GitHub

My aim to use a custom Word Delimiter Token Filter is to change
"stem_english_possessive" to false.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--

Regards,
Varun Thacker
http://www.vthacker.in/

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

I forgot to mention that \u0027 is json notation and when you use it in the
_analyze request body it's not getting converted to an apostrophe. You can
use ''' or quotes instead:

curl -XGET 'localhost:9200/test2/_analyze?analyzer=myAnalyzer&pretty=true'
-d 'world-class player wouldn'''t'

curl -XGET 'localhost:9200/test2/_analyze?analyzer=myAnalyzer&pretty=true'
-d "world-class player wouldn't"

On Tuesday, February 5, 2013 7:25:00 AM UTC-5, Igor Motov wrote:

That looks like a bug that was introduced during Lucene 4 migration. We
will fix it.

On Monday, February 4, 2013 6:12:41 AM UTC-5, Varun Thacker wrote:

Hi Martijn,

I'm using a build against master with all the latest commits.

I've posted the results which I am seeing in the comments section (
Word Delimiter Token Filter · GitHub)

On Mon, Feb 4, 2013 at 3:07 PM, Martijn v Groningen <
martijn.v...@gmail.com> wrote:

Hi Varun,

I don't see different behaviour between the two analyzer requests with
version 0.20.4
What version are you using?

Martijn

On 4 February 2013 06:20, Varun Thacker varuntha...@gmail.com wrote:

So I have 2 mappings one where I use the default Word Delimiter Token
Filter and one where I use a custom one but don't really change any of the
defaults. Both behave different behaviour. I'm not sure why this is
happening.

I've pasted the mappings here : Word Delimiter Token Filter · GitHub

My aim to use a custom Word Delimiter Token Filter is to change
"stem_english_possessive" to false.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Regards,
Varun Thacker
http://www.vthacker.in/

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.