# Adding analyzer to dynamic fields **URL:** \ **Category:** Elasticsearch\ **Created:** [April 29, 2013, 8:08am UTC](https://discuss.elastic.co/t/adding-analyzer-to-dynamic-fields/11727 "2013-04-29T08:08:34Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Onur\_Aktas](https://avatars.discourse-cdn.com/v4/letter/o/ac8455/32.png) [@Onur\_Aktas](https://discuss.elastic.co/u/Onur_Aktas)\ **Post date:** [April 29, 2013, 8:08am UTC](https://discuss.elastic.co/t/adding-analyzer-to-dynamic-fields/11727/1 "2013-04-29T08:08:34Z")
Hi, I'm new to ES and need some advice about mapping / adding analyzers to dynamic fields. The product document I want to Index in ES has some dynamic fields (fields starting with "f\_") based on the Product type. _Product A:_ { name: "Product A", f\_material: ["Material A", "Material B"], f\_type: "Type A" } _Product B:_ { name: "Product B", f\_gender: "Male", f\_size: ["M", "L", "XL", "XXL"] } As you can see Fields starting with f\_ are dynamic and change based on the product type. One has f\_material and f\_type while the other has f\_size, f\_gender. So; 1. How can I add comma separated analyzer to dynamic fields? Can we define analyzers to the field names that match some regex? (i.e "^f\_.+") 2. There may be around 200, 300 dynamically named fields, as I read from different topics I saw that they cause memory issues on Lucene. If I kept their names as "f\_1", "f\_2".. instead of "f\_material", "f\_type", would it prevent memory issues? Thanks for your advices, Onur -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Onur\_Aktas](https://avatars.discourse-cdn.com/v4/letter/o/ac8455/32.png) [@Onur\_Aktas](https://discuss.elastic.co/u/Onur_Aktas)\ **Post date:** [April 29, 2013, 9:26am UTC](https://discuss.elastic.co/t/adding-analyzer-to-dynamic-fields/11727/2 "2013-04-29T09:26:49Z")
Adding this as mapping type resolves my problem. dynamic\_templates: [ - { - template\_feature: { - mapping: { - type: multi\_field - fields: { - {name}: { - type: {dynamic\_type} - index: analyzed } - org: { - type: {dynamic\_type} - index: not\_analyzed } } } - match: f\_\* } } ] On Monday, April 29, 2013 11:08:34 AM UTC+3, Onur Aktaş wrote: > Hi, > > I'm new to ES and need some advice about mapping / adding analyzers to > dynamic fields. The product document I want to Index in ES has some dynamic > fields (fields starting with "f\_") based on the Product type. > > _Product A:_ > { > name: "Product A", > f\_material: ["Material A", "Material B"], > f\_type: "Type A" > } > > _Product B:_ > { > name: "Product B", > f\_gender: "Male", > f\_size: ["M", "L", "XL", "XXL"] > } > > As you can see Fields starting with f\_ are dynamic and change based on the > product type. One has f\_material and f\_type while the other has f\_size, > f\_gender. > So; > > 1. How can I add comma separated analyzer to dynamic fields? Can we > define analyzers to the field names that match some regex? (i.e "^f\_.+") > 2. There may be around 200, 300 dynamically named fields, as I read > from different topics I saw that they cause memory issues on Lucene. If I > kept their names as "f\_1", "f\_2".. instead of "f\_material", "f\_type", > would it prevent memory issues? > > Thanks for your advices, > Onur -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [April 29, 2013, 2:19pm UTC](https://discuss.elastic.co/t/adding-analyzer-to-dynamic-fields/11727/3 "2013-04-29T14:19:59Z")
If you are worried about memory issues, are you sure you want to have a multi field on every dynamic property? In your example, your values are multi-valued and are not a single value that is comma delimited (the correct way to do things). Are you sure you need a custom analyzer? -- Ivan On Mon, Apr 29, 2013 at 2:26 AM, Onur Aktaş [onurraktas@gmail.com](mailto:onurraktas@gmail.com) wrote: > Adding this as mapping type resolves my problem. > > dynamic\_templates: [ > > - { > - template\_feature: { > - mapping: { > - type: multi\_field > - fields: { > - {name}: { > - type: {dynamic\_type} > - index: analyzed > } > > - org: { > - type: {dynamic\_type} > - index: not\_analyzed > } > } > } > > - match: f\_\* > } > } > > ] > > On Monday, April 29, 2013 11:08:34 AM UTC+3, Onur Aktaş wrote: > > > Hi, > > > > I'm new to ES and need some advice about mapping / adding analyzers to > > dynamic fields. The product document I want to Index in ES has some dynamic > > fields (fields starting with "f\_") based on the Product type. > > > > _Product A:_ > > { > > name: "Product A", > > f\_material: ["Material A", "Material B"], > > f\_type: "Type A" > > } > > > > _Product B:_ > > { > > name: "Product B", > > f\_gender: "Male", > > f\_size: ["M", "L", "XL", "XXL"] > > } > > > > As you can see Fields starting with f\_ are dynamic and change based on > > the product type. One has f\_material and f\_type while the other has f\_size, > > f\_gender. > > So; > > > > 1. How can I add comma separated analyzer to dynamic fields? Can we > > define analyzers to the field names that match some regex? (i.e "^f\_.+") > > 2. There may be around 200, 300 dynamically named fields, as I read > > from different topics I saw that they cause memory issues on Lucene. If I > > kept their names as "f\_1", "f\_2".. instead of "f\_material", "f\_type", > > would it prevent memory issues? > > > > Thanks for your advices, > > Onur > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Onur\_Aktas](https://avatars.discourse-cdn.com/v4/letter/o/ac8455/32.png) [@Onur\_Aktas](https://discuss.elastic.co/u/Onur_Aktas)\ **Post date:** [April 30, 2013, 11:32am UTC](https://discuss.elastic.co/t/adding-analyzer-to-dynamic-fields/11727/4 "2013-04-30T11:32:02Z")
With multi-field I was able to get facets by"f\_type.org", "f\_material.org" etc, however they didn't work for comma separated multi-values as you said. Could you please tell how I can achieve this without creating multi field on every dynamic property?. All I want to do is to get facets based on dynamic fields without breaking them into words except the ones containg comma. Material: - Material A (5) - Material B (10) Size: - L(1) - XL(5) - XL(10) instead of Material: - Material (5) - A (5) - B (5) Onur On Monday, April 29, 2013 5:19:59 PM UTC+3, Ivan Brusic wrote: > If you are worried about memory issues, are you sure you want to have a > multi field on every dynamic property? In your example, your values are > multi-valued and are not a single value that is comma delimited (the > correct way to do things). Are you sure you need a custom analyzer? > > -- > Ivan > > On Mon, Apr 29, 2013 at 2:26 AM, Onur Aktaş \<[onurr...@gmail.com](mailto:onurr...@gmail.com)\ > > > wrote: > > > Adding this as mapping type resolves my problem. > > > > dynamic\_templates: [ > > > > - { > > - template\_feature: { > > - mapping: { > > - type: multi\_field > > - fields: { > > - {name}: { > > - type: {dynamic\_type} > > - index: analyzed > > } > > > > - org: { > > - type: {dynamic\_type} > > - index: not\_analyzed > > } > > } > > } > > > > - match: f\_\* > > } > > } > > > > ] > > > > On Monday, April 29, 2013 11:08:34 AM UTC+3, Onur Aktaş wrote: > > > > > Hi, > > > > > > I'm new to ES and need some advice about mapping / adding analyzers to > > > dynamic fields. The product document I want to Index in ES has some dynamic > > > fields (fields starting with "f\_") based on the Product type. > > > > > > _Product A:_ > > > { > > > name: "Product A", > > > f\_material: ["Material A", "Material B"], > > > f\_type: "Type A" > > > } > > > > > > _Product B:_ > > > { > > > name: "Product B", > > > f\_gender: "Male", > > > f\_size: ["M", "L", "XL", "XXL"] > > > } > > > > > > As you can see Fields starting with f\_ are dynamic and change based on > > > the product type. One has f\_material and f\_type while the other has f\_size, > > > f\_gender. > > > So; > > > > > > 1. How can I add comma separated analyzer to dynamic fields? Can we > > > define analyzers to the field names that match some regex? (i.e "^f\_.+") > > > 2. There may be around 200, 300 dynamically named fields, as I read > > > from different topics I saw that they cause memory issues on Lucene. If I > > > kept their names as "f\_1", "f\_2".. instead of "f\_material", "f\_type", > > > would it prevent memory issues? > > > > > > Thanks for your advices, > > > Onur > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [April 30, 2013, 4:27pm UTC](https://discuss.elastic.co/t/adding-analyzer-to-dynamic-fields/11727/5 "2013-04-30T16:27:40Z")
To me, it sounds like the processing needs to happen once at index time, unless you need the original value to be returned. Your client-side indexing process can do the comma tokenization process and you can just index the tokens as non-analyzed. Or you can setup a index analyzer which knows how to process the text and set your search analyzer as a simple KeywordAnalyzer (identical to using not analyzed). -- Ivan On Tue, Apr 30, 2013 at 4:32 AM, Onur Aktaş [onurraktas@gmail.com](mailto:onurraktas@gmail.com) wrote: > With multi-field I was able to get facets by"f\_type.org", "f\_material.org" > etc, however they didn't work for comma separated multi-values as you said. > Could you please tell how I can achieve this without creating multi field > on every dynamic property?. > > All I want to do is to get facets based on dynamic fields without breaking > them into words except the ones containg comma. > > Material: > > - Material A (5) > - Material B (10) > > Size: > > - L(1) > - XL(5) > - XL(10) > > instead of > > Material: > > - Material (5) > - A (5) > - B (5) > > Onur > > On Monday, April 29, 2013 5:19:59 PM UTC+3, Ivan Brusic wrote: > > > If you are worried about memory issues, are you sure you want to have a > > multi field on every dynamic property? In your example, your values are > > multi-valued and are not a single value that is comma delimited (the > > correct way to do things). Are you sure you need a custom analyzer? > > > > -- > > Ivan > > > > On Mon, Apr 29, 2013 at 2:26 AM, Onur Aktaş [onurr...@gmail.com](mailto:onurr...@gmail.com) wrote: > > > > > Adding this as mapping type resolves my problem. > > > > > > dynamic\_templates: [ > > > > > > - { > > > - template\_feature: { > > > - mapping: { > > > - type: multi\_field > > > - fields: { > > > - {name}: { > > > - type: {dynamic\_type} > > > - index: analyzed > > > } > > > > > > - org: { > > > - type: {dynamic\_type} > > > - index: not\_analyzed > > > } > > > } > > > } > > > > > > - match: f\_\* > > > } > > > } > > > > > > ] > > > > > > On Monday, April 29, 2013 11:08:34 AM UTC+3, Onur Aktaş wrote: > > > > > > > Hi, > > > > > > > > I'm new to ES and need some advice about mapping / adding analyzers to > > > > dynamic fields. The product document I want to Index in ES has some dynamic > > > > fields (fields starting with "f\_") based on the Product type. > > > > > > > > _Product A:_ > > > > { > > > > name: "Product A", > > > > f\_material: ["Material A", "Material B"], > > > > f\_type: "Type A" > > > > } > > > > > > > > _Product B:_ > > > > { > > > > name: "Product B", > > > > f\_gender: "Male", > > > > f\_size: ["M", "L", "XL", "XXL"] > > > > } > > > > > > > > As you can see Fields starting with f\_ are dynamic and change based on > > > > the product type. One has f\_material and f\_type while the other has f\_size, > > > > f\_gender. > > > > So; > > > > > > > > 1. How can I add comma separated analyzer to dynamic fields? Can we > > > > define analyzers to the field names that match some regex? (i.e "^f\_.+") > > > > 2. There may be around 200, 300 dynamically named fields, as I read > > > > from different topics I saw that they cause memory issues on Lucene. If I > > > > kept their names as "f\_1", "f\_2".. instead of "f\_material", "f\_type", > > > > would it prevent memory issues? > > > > > > > > Thanks for your advices, > > > > Onur > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com). > > > > > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > > . > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:39am UTC](https://discuss.elastic.co/t/adding-analyzer-to-dynamic-fields/11727/6 "2017-07-06T02:39:03Z")