# How to use one token filter for indexing and another for search? **URL:** \ **Category:** Elasticsearch\ **Created:** [October 25, 2013, 11:32am UTC](https://discuss.elastic.co/t/how-to-use-one-token-filter-for-indexing-and-another-for-search/14114 "2013-10-25T11:32:15Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Petr\_Jansky](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Petr\_Jansky](https://discuss.elastic.co/u/Petr_Jansky)\ **Post date:** [October 25, 2013, 11:32am UTC](https://discuss.elastic.co/t/how-to-use-one-token-filter-for-indexing-and-another-for-search/14114/1 "2013-10-25T11:32:15Z")
Hello, I would like to use two different token filters for same field(s): 1st - for indexing that creates terms: - stem - sentence elements(noun, adjective...) - verb conjugations and noun declensions and adjective - ..... 2nd - for search that retrieves only: - stem (the same as the 1st filter) I want to be able to use classic search and search specific sentence elements using curl -X GET 'localhost:9200/\_search?pretty=true' -d '{ "query" : { "term" : { "content" : "==noun==" } }, "highlight" : { "tags\_schema" : "styled", "fields" : { "content" : {} } } }' I know it's very unusual to use an different token filter for search than for indexing. But I think it's better way than duplicate index fields for each token filter. Thanks Petr -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [October 25, 2013, 11:46am UTC](https://discuss.elastic.co/t/how-to-use-one-token-filter-for-indexing-and-another-for-search/14114/2 "2013-10-25T11:46:12Z")
Hi Petr Create two custom analyzers in your index, then you can set fields to use one for the `search_analyzer` and the other for the `index_analyzer` clint On 25 October 2013 13:32, Petr Janský [petr.jansky@6hats.cz](mailto:petr.jansky@6hats.cz) wrote: > Hello, > > I would like to use two different token filters for same field(s): > 1st - for indexing that creates terms: > > - stem > - sentence elements(noun, adjective...) > - verb conjugations and noun declensions and adjective > - ..... > > 2nd - for search that retrieves only: > > - stem (the same as the 1st filter) > > I want to be able to use classic search and search specific sentence > elements using > > curl -X GET 'localhost:9200/\_search?pretty=true' -d '{ > "query" : { > "term" : { "content" : "==noun==" } > }, > "highlight" : { > "tags\_schema" : "styled", > "fields" : { > "content" : {} > } > } > }' > > I know it's very unusual to use an different token filter for search than > for indexing. But I think it's better way than duplicate index fields for > each token filter. > > Thanks > Petr > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [October 25, 2013, 11:46am UTC](https://discuss.elastic.co/t/how-to-use-one-token-filter-for-indexing-and-another-for-search/14114/3 "2013-10-25T11:46:44Z")
Alternatively, you can always specify a particular `analyzer` at query time in the query itself On 25 October 2013 13:46, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com) wrote: > Hi Petr > > Create two custom analyzers in your index, then you can set fields to use > one for the `search_analyzer` and the other for the `index_analyzer` > > clint > > On 25 October 2013 13:32, Petr Janský [petr.jansky@6hats.cz](mailto:petr.jansky@6hats.cz) wrote: > > > Hello, > > > > I would like to use two different token filters for same field(s): > > 1st - for indexing that creates terms: > > > > - stem > > - sentence elements(noun, adjective...) > > - verb conjugations and noun declensions and adjective > > - ..... > > > > 2nd - for search that retrieves only: > > > > - stem (the same as the 1st filter) > > > > I want to be able to use classic search and search specific sentence > > elements using > > > > curl -X GET 'localhost:9200/\_search?pretty=true' -d '{ > > "query" : { > > "term" : { "content" : "==noun==" } > > }, > > "highlight" : { > > "tags\_schema" : "styled", > > "fields" : { > > "content" : {} > > } > > } > > }' > > > > I know it's very unusual to use an different token filter for search than > > for indexing. But I think it's better way than duplicate index fields for > > each token filter. > > > > Thanks > > Petr > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Petr\_Jansky](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Petr\_Jansky](https://discuss.elastic.co/u/Petr_Jansky)\ **Post date:** [October 25, 2013, 11:56am UTC](https://discuss.elastic.co/t/how-to-use-one-token-filter-for-indexing-and-another-for-search/14114/4 "2013-10-25T11:56:03Z")
Hi Clint, thank you for your quick reply. So if I get you right I should change my analyzers and mapping to _curl -X POST 'localhost:9200/index3/' -d '{_ - "settings" : {\* - "analysis" : {\* - ``` "analyzer" : {* ``` - ``` "index_cestina" : {* ``` - ``` "type" : "custom",* ``` - ``` "tokenizer" : "standard",* ``` - ``` "filter" : ["stopwords_CZ", "index_mor_czech"]* ``` - ``` },* ``` - "search\_cestina" : {\* - ``` "type" : "custom",* ``` - ``` "tokenizer" : "standard",* ``` - ``` "filter" : ["stopwords_CZ", "search_mor_czech"]* ``` - ``` }* ``` - ``` },* ``` - "stopwords\_CZ" : {\* - ``` "type" : "stop",* ``` - ``` "stopwords" : ["právě", "že", "_czech_"],* ``` - ``` "ignore_case" : true* ``` - ``` },* ``` - ``` "filter" : {* ``` - ``` "index_mor_czech" : {* ``` - ``` "type" : "morphologyIndex",* ``` - ``` "language" : "cs",* ``` - ``` "path" : "/opt/morphologyCs"* ``` - },\* - "search\_mor\_czech" : {\* - ``` "type" : "morphologySearch",* ``` - ``` "language" : "cs",* ``` - ``` "path" : "/opt/morphologyCs"* ``` - } \* - }}},\* - "mappings" : {\* - "article" : {\* - ``` "_id" : {* ``` - ``` "path" : "reference"* ``` - ``` },* ``` - "properties" : {\* - ``` "title" : { "type" : "string", "index_analyzer" : ``` "index\_cestina", "search\_analyzer" : "search\_cestina"},\* - ``` "content" : { "type" : "string", "index_analyzer" : ``` "index\_cestina", "search\_analyzer" : "search\_cestina"}\* _}}}}'_ Thanks Petr Dne pátek, 25. října 2013 13:46:44 UTC+2 Clinton Gormley napsal(a): > Alternatively, you can always specify a particular `analyzer` at query > time in the query itself > > On 25 October 2013 13:46, Clinton Gormley \<[cl...@traveljury.com](mailto:cl...@traveljury.com)\ > > > wrote: > > > Hi Petr > > > > Create two custom analyzers in your index, then you can set fields to use > > one for the `search_analyzer` and the other for the `index_analyzer` > > > > clint > > > > On 25 October 2013 13:32, Petr Janský \\>wrote: > > > > > Hello, > > > > > > I would like to use two different token filters for same field(s): > > > 1st - for indexing that creates terms: > > > > > > - stem > > > - sentence elements(noun, adjective...) > > > - verb conjugations and noun declensions and adjective > > > - ..... > > > > > > 2nd - for search that retrieves only: > > > > > > - stem (the same as the 1st filter) > > > > > > I want to be able to use classic search and search specific sentence > > > elements using > > > > > > curl -X GET 'localhost:9200/\_search?pretty=true' -d '{ > > > "query" : { > > > "term" : { "content" : "==noun==" } > > > }, > > > "highlight" : { > > > "tags\_schema" : "styled", > > > "fields" : { > > > "content" : {} > > > } > > > } > > > }' > > > > > > I know it's very unusual to use an different token filter for search > > > than for indexing. But I think it's better way than duplicate index fields > > > for each token filter. > > > > > > Thanks > > > Petr > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [October 27, 2013, 11:49am UTC](https://discuss.elastic.co/t/how-to-use-one-token-filter-for-indexing-and-another-for-search/14114/5 "2013-10-27T11:49:39Z")
Hi Petr, Your example is looking good to me. I was testing search/index analyzers for which I created the following small recreation script: > (It is testing that the search\_analyzer is used and it works fine). Out of curiosity, is there any public record about what the morphologyIndex token filter type is based on? Is it based on @imotov's analysis-morphology plugin or some other (proprietary?) solution? Thanks, Lukáš On Fri, Oct 25, 2013 at 6:56 AM, Petr Janský [petr.jansky@6hats.cz](mailto:petr.jansky@6hats.cz) wrote: > Hi Clint, > > thank you for your quick reply. > > So if I get you right I should change my analyzers and mapping to > > _curl -X POST 'localhost:9200/index3/' -d '{_ > > - "settings" : {\* > - "analysis" : {\* > - > > ``` > "analyzer" : {* > > ``` > > - > > ``` > "index_cestina" : {* > > ``` > > - > > ``` > "type" : "custom",* > > ``` > > - > > ``` > "tokenizer" : "standard",* > > ``` > > - > > ``` > "filter" : ["stopwords_CZ", "index_mor_czech"]* > > ``` > > - > > ``` > },* > > ``` > > - "search\_cestina" : {\* > - > > ``` > "type" : "custom",* > > ``` > > - > > ``` > "tokenizer" : "standard",* > > ``` > > - > > ``` > "filter" : ["stopwords_CZ", "search_mor_czech"]* > > ``` > > - > > ``` > }* > > ``` > > - > > ``` > },* > > ``` > > - "stopwords\_CZ" : {\* > - > > ``` > "type" : "stop",* > > ``` > > - > > ``` > "stopwords" : ["právě", "že", "_czech_"],* > > ``` > > - > > ``` > "ignore_case" : true* > > ``` > > - > > ``` > },* > > ``` > > - > > ``` > "filter" : {* > > ``` > > - > > ``` > "index_mor_czech" : {* > > ``` > > - > > ``` > "type" : "morphologyIndex",* > > ``` > > - > > ``` > "language" : "cs",* > > ``` > > - > > ``` > "path" : "/opt/morphologyCs"* > > ``` > > - },\* > - "search\_mor\_czech" : {\* > - > > ``` > "type" : "morphologySearch",* > > ``` > > - > > ``` > "language" : "cs",* > > ``` > > - > > ``` > "path" : "/opt/morphologyCs"* > > ``` > > - } \* > - }}},\* > - "mappings" : {\* > - "article" : {\* > - > > ``` > "_id" : {* > > ``` > > - > > ``` > "path" : "reference"* > > ``` > > - > > ``` > },* > > ``` > > - "properties" : {\* > - > > ``` > "title" : { "type" : "string", "index_analyzer" : > > ``` > > "index\_cestina", "search\_analyzer" : "search\_cestina"},\* > > - > > ``` > "content" : { "type" : "string", "index_analyzer" : > > ``` > > "index\_cestina", "search\_analyzer" : "search\_cestina"}\* > _}}}}'_ > > Thanks > Petr > > Dne pátek, 25. října 2013 13:46:44 UTC+2 Clinton Gormley napsal(a): > > > Alternatively, you can always specify a particular `analyzer` at query > > time in the query itself > > > > On 25 October 2013 13:46, Clinton Gormley [cl...@traveljury.com](mailto:cl...@traveljury.com) wrote: > > > > > Hi Petr > > > > > > Create two custom analyzers in your index, then you can set fields to > > > use one for the `search_analyzer` and the other for the `index_analyzer` > > > > > > clint > > > > > > On 25 October 2013 13:32, Petr Janský [petr....@6hats.cz](mailto:petr....@6hats.cz) wrote: > > > > > > > Hello, > > > > > > > > I would like to use two different token filters for same field(s): > > > > 1st - for indexing that creates terms: > > > > > > > > - stem > > > > - sentence elements(noun, adjective...) > > > > - verb conjugations and noun declensions and adjective > > > > - ..... > > > > > > > > 2nd - for search that retrieves only: > > > > > > > > - stem (the same as the 1st filter) > > > > > > > > I want to be able to use classic search and search specific sentence > > > > elements using > > > > > > > > curl -X GET 'localhost:9200/\_search?\*\*pretty=true' -d '{ > > > > "query" : { > > > > "term" : { "content" : "==noun==" } > > > > }, > > > > "highlight" : { > > > > "tags\_schema" : "styled", > > > > "fields" : { > > > > "content" : {} > > > > } > > > > } > > > > }' > > > > > > > > I know it's very unusual to use an different token filter for search > > > > than for indexing. But I think it's better way than duplicate index fields > > > > for each token filter. > > > > > > > > Thanks > > > > Petr > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com). > > > > > > > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > > > . > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:10am UTC](https://discuss.elastic.co/t/how-to-use-one-token-filter-for-indexing-and-another-for-search/14114/6 "2017-07-06T02:10:26Z")