# Can I exclude certain fields from being stemmed when rolled into a stemmed composite field via multifield? Can Keyword Filter / Stemmer Override be used? **URL:** \ **Category:** Elasticsearch\ **Created:** [March 25, 2013, 9:47pm UTC](https://discuss.elastic.co/t/can-i-exclude-certain-fields-from-being-stemmed-when-rolled-into-a-stemmed-composite-field-via-multifield-can-keyword-filter-stemmer-override-be-used/11305 "2013-03-25T21:47:39Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![roytmana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/roytmana/32/44855_2.png) [@roytmana](https://discuss.elastic.co/u/roytmana)\ **Post date:** [March 25, 2013, 9:47pm UTC](https://discuss.elastic.co/t/can-i-exclude-certain-fields-from-being-stemmed-when-rolled-into-a-stemmed-composite-field-via-multifield-can-keyword-filter-stemmer-override-be-used/11305/1 "2013-03-25T21:47:39Z")
Hello, I would like to prevent certain fields that are fed into "\_all" or some custom composite multifield that has stemming enabled from being stemmed. I do not know exactly how multifield and \_all work. Are tokens generated on original fields get duplicated and channeled to \_all or multifields? If so is there any way to inject token filter before it is channeled to \_all/multifield If so is it possible to use a token filter to mark them as keywords so they do not get stemmed when added to \_all? What's the difference between those two filters: They seem to do the same - protect explicetly listed words from stemming by applying Keyword flag Thank you, Alex -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![roytmana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/roytmana/32/44855_2.png) [@roytmana](https://discuss.elastic.co/u/roytmana)\ **Post date:** [March 26, 2013, 2:21pm UTC](https://discuss.elastic.co/t/can-i-exclude-certain-fields-from-being-stemmed-when-rolled-into-a-stemmed-composite-field-via-multifield-can-keyword-filter-stemmer-override-be-used/11305/2 "2013-03-26T14:21:29Z")
Bump - Any suggestions? On Monday, March 25, 2013 5:47:39 PM UTC-4, AlexR wrote: > Hello, > > I would like to prevent certain fields that are fed into "\_all" or some > custom composite multifield that has stemming enabled from being stemmed. > > I do not know exactly how multifield and \_all work. Are tokens generated > on original fields get duplicated and channeled to \_all or multifields? If > so is there any way to inject token filter before it is channeled to > \_all/multifield > If so is it possible to use a token filter to mark them as keywords so > they do not get stemmed when added to \_all? > > What's the difference between those two filters: > They seem to do the same - protect explicetly listed words from stemming > by applying Keyword flag > > Thank you, > Alex -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 26, 2013, 2:32pm UTC](https://discuss.elastic.co/t/can-i-exclude-certain-fields-from-being-stemmed-when-rolled-into-a-stemmed-composite-field-via-multifield-can-keyword-filter-stemmer-override-be-used/11305/3 "2013-03-26T14:32:16Z")
The keyword analyzer takes input as it is (think of an "input stream identity analyzer") . You can easily set up multifield to add other analyzer to the same input of a field. Multifield branches the input to many fields. Each field carries its analyzers, and so are tokens generated for the fields. Note also \_all is a field with an analyzer that can be defined. But you will get weird result for keyword analyzer in \_all because \_all is a concatentation of many fields - you will notice that querying such a field set to keyword analyzer is close to impossible. Note that there is also word separation and other algorithms in an analyzer. I hope this helps. Just try multifield with ES to find out how it works. There is much power behind it, in combination with "analyzer" and "index". Jörg Am 25.03.13 22:47, schrieb AlexR: > Hello, > > I would like to prevent certain fields that are fed into "\_all" or > some custom composite multifield that has stemming enabled from being > stemmed. > > I do not know exactly how multifield and \_all work. Are tokens > generated on original fields get duplicated and channeled to \_all or > multifields? If so is there any way to inject token filter before it > is channeled to \_all/multifield > If so is it possible to use a token filter to mark them as keywords so > they do not get stemmed when added to \_all? > > What's the difference between those two filters: > They seem to do the same - protect explicetly listed words from > stemming by applying Keyword flag > > ## Thank you, Alex > > You received this message because you are subscribed to the Google > Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![roytmana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/roytmana/32/44855_2.png) [@roytmana](https://discuss.elastic.co/u/roytmana)\ **Post date:** [March 26, 2013, 3:38pm UTC](https://discuss.elastic.co/t/can-i-exclude-certain-fields-from-being-stemmed-when-rolled-into-a-stemmed-composite-field-via-multifield-can-keyword-filter-stemmer-override-be-used/11305/4 "2013-03-26T15:38:35Z")
Thanks Jörg, I use multifields extensively. I think I did not express my question clearly. Say I have object Memo with two fields companyName and text. I want a multifield called memo\_search which will combine both fields and it will be stemmed retaining original terms like this ["standard", "lowercase", "stop", "stem\_possessive\_english", "keyword\_repeat", "kstem", "stem\_deduplicate"] but I would like companyName in memo\_search field to stay unstemmed by giving its tokens a keyword flag to make stemmer of memo\_search of to ignore it (stemmers do not modify any token marked as a keyword). Keyword Filter and Stemmer Override use this strategy to preserve predefined tokens from stemming) So my question is: 1. When I define a multifield mapping will ES take value of the field and add it to my multifield or will it copy tokens produced during processing of the primary field and send them to secondary multifield(s)? 2. If answer in #1 is tokens, is there any way I can mark all tokens as keywords so stemmer in memo\_search will pass them unchanged 3. If not, Is there any way to specify additional filters on a secondary multifield so they will be applied prior to filters defined for the multifield itself (memo\_search) so I can mark them as keywords Now I know you will say why not collect my memo fields in two multifields - one stemmed and one not with additional benefit of boosting unstemmed field in searches. Yes it would be great if it we just handful of fields but I have a deeply nested JSON with couple of hundreds of fileds and besides stemming I want shingles so mappings grow like a snowball and become very error prone on change so I am exploring the alternative option. This option of course has one potential issue - I will be running stemmed query over a mix of stemmed and unstemmed text which could give me some false positives on unstemed data (i.e. searching for "running" it stemmed to "run" and "running" in query (I retain original values in my analyzer) and I get match on "Run for life Inc" company name - undesired but so I will have it with two stemmed/unstemmed fields except for custom bust I could use with two fields On Tuesday, March 26, 2013 10:32:16 AM UTC-4, Jörg Prante wrote: > The keyword analyzer takes input as it is (think of an "input stream > identity analyzer") . > > You can easily set up multifield to add other analyzer to the same input > of a field. Multifield branches the input to many fields. Each field > carries its analyzers, and so are tokens generated for the fields. Note > also \_all is a field with an analyzer that can be defined. But you will > get weird result for keyword analyzer in \_all because \_all is a > concatentation of many fields - you will notice that querying such a > field set to keyword analyzer is close to impossible. > > Note that there is also word separation and other algorithms in an > analyzer. > > I hope this helps. Just try multifield with ES to find out how it works. > There is much power behind it, in combination with "analyzer" and "index". > > Jörg > > Am 25.03.13 22:47, schrieb AlexR: > > > Hello, > > > > I would like to prevent certain fields that are fed into "\_all" or > > some custom composite multifield that has stemming enabled from being > > stemmed. > > > > I do not know exactly how multifield and \_all work. Are tokens > > generated on original fields get duplicated and channeled to \_all or > > multifields? If so is there any way to inject token filter before it > > is channeled to \_all/multifield > > If so is it possible to use a token filter to mark them as keywords so > > they do not get stemmed when added to \_all? > > > > What's the difference between those two filters: > > They seem to do the same - protect explicetly listed words from > > stemming by applying Keyword flag > > > > ## Thank you, Alex > > > > You received this message because you are subscribed to the Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:44am UTC](https://discuss.elastic.co/t/can-i-exclude-certain-fields-from-being-stemmed-when-rolled-into-a-stemmed-composite-field-via-multifield-can-keyword-filter-stemmer-override-be-used/11305/5 "2017-07-06T02:44:23Z")