How to use ngram tokenizer for words that output from synonym filter?


(Zhishan Zhao) #1

hi, I want use standard tokenizer to split sentence into words first, then use synonym filter to normalize synonyms, and finally use ngram tokenizer to break the words that output from synonym filter into characters。In simple terms:

standard tokenizer -> split sentences
synonym filter -> map the synonyms to a single word,
ngram tokenizer -> break the words that output from synonym filter into characters

Is there any way to do this?


(David Pilato) #2

No. Not exactly.

You need to use a ngram tokenfilter instead.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.