Wrapping TokenFilters

Elastic Stack Elasticsearch
1

Hi,

I'm trying to write a TokenFilter plugin that will wrap an existing TokenFilter (in this case, a stemmer), and I'm having trouble working out how to get access to that existing filter in the plugin's TokenFilterFactory. Does anyone have any suggestions? I've tried passing in an IndicesAnalysisService, but that doesn't seem to contain the standard stemmer filters.

Thanks,

Alan Woodward

--

2

TokenFilters are operating in a chain, so there is rarely a need to wrap
one filter into another because you can simply add your filter before or
after the filter that you are trying to wrap. If you really really need to
wrap it, you can simply create a new stemmer.

On Thursday, January 24, 2013 6:44:49 AM UTC-5, Alan Woodward wrote:

Hi,

I'm trying to write a TokenFilter plugin that will wrap an existing
TokenFilter (in this case, a stemmer), and I'm having trouble working out
how to get access to that existing filter in the plugin's
TokenFilterFactory. Does anyone have any suggestions? I've tried passing
in an IndicesAnalysisService, but that doesn't seem to contain the standard
stemmer filters.

Thanks,

Alan Woodward
www.flax.co.uk

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hey Alan,

can you do a simple cast in the wrapper and configure the token filter so
it gets injected as the input TokenStream?

simon

On Thursday, January 24, 2013 12:44:49 PM UTC+1, Alan Woodward wrote:

Hi,

I'm trying to write a TokenFilter plugin that will wrap an existing
TokenFilter (in this case, a stemmer), and I'm having trouble working out
how to get access to that existing filter in the plugin's
TokenFilterFactory. Does anyone have any suggestions? I've tried passing
in an IndicesAnalysisService, but that doesn't seem to contain the standard
stemmer filters.

Thanks,

Alan Woodward
www.flax.co.uk

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

I solved this without doing any wrapping at all in the end, by inserting a TokenFilter into the stream that added KeywordAttributes to the relevant tokens, so that the stemmer would then ignore them.

Thanks for the suggestions, though!

Alan Woodward

On 2 Feb 2013, at 20:11, simonw wrote:

Hey Alan,

can you do a simple cast in the wrapper and configure the token filter so it gets injected as the input TokenStream?

simon

On Thursday, January 24, 2013 12:44:49 PM UTC+1, Alan Woodward wrote:
Hi,

I'm trying to write a TokenFilter plugin that will wrap an existing TokenFilter (in this case, a stemmer), and I'm having trouble working out how to get access to that existing filter in the plugin's TokenFilterFactory. Does anyone have any suggestions? I've tried passing in an IndicesAnalysisService, but that doesn't seem to contain the standard stemmer filters.

Thanks,

Alan Woodward
www.flax.co.uk

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

yeah keywordattr is useful though!

thanks for reporting back

On Sunday, February 3, 2013 11:18:20 AM UTC+1, Alan Woodward wrote:

I solved this without doing any wrapping at all in the end, by inserting a
TokenFilter into the stream that added KeywordAttributes to the relevant
tokens, so that the stemmer would then ignore them.

Thanks for the suggestions, though!

Alan Woodward
www.flax.co.uk

On 2 Feb 2013, at 20:11, simonw wrote:

Hey Alan,

can you do a simple cast in the wrapper and configure the token filter so
it gets injected as the input TokenStream?

simon

On Thursday, January 24, 2013 12:44:49 PM UTC+1, Alan Woodward wrote:

Hi,

I'm trying to write a TokenFilter plugin that will wrap an existing
TokenFilter (in this case, a stemmer), and I'm having trouble working out
how to get access to that existing filter in the plugin's
TokenFilterFactory. Does anyone have any suggestions? I've tried passing
in an IndicesAnalysisService, but that doesn't seem to contain the standard
stemmer filters.

Thanks,

Alan Woodward
www.flax.co.uk

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.