# Creating dynamic fields from a field **URL:** \ **Category:** Elasticsearch\ **Created:** [March 14, 2014, 8:20pm UTC](https://discuss.elastic.co/t/creating-dynamic-fields-from-a-field/16383 "2014-03-14T20:20:00Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Pablo\_Musa](https://avatars.discourse-cdn.com/v4/letter/p/34f0e0/32.png) [@Pablo\_Musa](https://discuss.elastic.co/u/Pablo_Musa)\ **Post date:** [March 14, 2014, 8:20pm UTC](https://discuss.elastic.co/t/creating-dynamic-fields-from-a-field/16383/1 "2014-03-14T20:20:00Z")
Hey guys, I have the following problem: Given a title, I want to record that title but I also want to record two fields for each word in the title, using the words itself as part of the field name. For example: given the title "The greatest band ever - Urban Legion" I would like to have a, document like: { "title":"The greatest band ever - Urban Legion", "greatest\_x" : 1, "band\_x" : 1, "ever\_x" : 1, "Urban\_x": 1, "Legion\_x" : 1, "greatest\_y" : [], "band\_y" : [], "ever\_y" : [], "Urban\_y": [], "Legion\_y" : [] } I was reading about dynamic mapping but I am not sure if I can acomplish the above. Is it possible to do inside ES? I could easily do it in an auxiliary application. Thanks, Pablo -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAF6PhFJisN\_puhY0xq-tvRe9gx0jLRReheRzjL9n\_PfxFAZ7pQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAF6PhFJisN_puhY0xq-tvRe9gx0jLRReheRzjL9n_PfxFAZ7pQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Binh\_Ly\_2](https://avatars.discourse-cdn.com/v4/letter/b/d07c76/32.png) [@Binh\_Ly\_2](https://discuss.elastic.co/u/Binh_Ly_2)\ **Post date:** [March 14, 2014, 8:58pm UTC](https://discuss.elastic.co/t/creating-dynamic-fields-from-a-field/16383/2 "2014-03-14T20:58:06Z")
No I don't believe you can automatically create fields based on the token values of another field. You'd probably have to do this outside of ES. If it matters, you can call the \_analyze API to produce the tokens before you inject your fields. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/ae093293-b65f-4661-b54f-e614accae5bb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ae093293-b65f-4661-b54f-e614accae5bb%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [March 15, 2014, 11:59am UTC](https://discuss.elastic.co/t/creating-dynamic-fields-from-a-field/16383/3 "2014-03-15T11:59:09Z")
To add to what Binh said, you really shouldn't add field names like this: On 14 March 2014 21:20, Pablo Musa [pablitomusa@gmail.com](mailto:pablitomusa@gmail.com) wrote: > { > "title":"The greatest band ever - Urban Legion", > "greatest\_x" : 1, > "band\_x" : 1, > "ever\_x" : 1, > "Urban\_x": 1, > "Legion\_x" : 1, > "greatest\_y" : , > "band\_y" : , > "ever\_y" : , > "Urban\_y": , > "Legion\_y" : > } You end up with an explosion of fields, and each field has an inverted index associated with it. Your cluster state will eventually become enormous. Any change to the cluster state (eg adding a field, changing an index, changes to nodes etc) results in the cluster state being copied to every node in the cluster. If the state is very large you will experience a significant slow down. Instead of: [{ custom\_foo: xxx }, { custom\_bar: yyy }] Use nested fields with eg [ { type: "custom\_foo", value: "xxx" }, { type: "custom\_bar", value: "yyy" } ] That way you have only two fields. clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSwR4p4pi\_z9cbS%3DLXcn-%3Dd\_OnzKzJM9P8MiXK\_4xpumg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSwR4p4pi_z9cbS%3DLXcn-%3Dd_OnzKzJM9P8MiXK_4xpumg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Pablo\_Musa](https://avatars.discourse-cdn.com/v4/letter/p/34f0e0/32.png) [@Pablo\_Musa](https://discuss.elastic.co/u/Pablo_Musa)\ **Post date:** [March 17, 2014, 7:11pm UTC](https://discuss.elastic.co/t/creating-dynamic-fields-from-a-field/16383/4 "2014-03-17T19:11:08Z")
Thank you very much for the hints 😉 > If it matters, you can call the \_analyze API to produce the tokens before > you inject your fields. Is there an URL there I can call? Or only using the internal API? > Your cluster state will eventually become enormous. Yes, I saw it coming but was postponing in dev phase. Thanks for the solution, it will help very much!! Thanks again guys! --Pablo On Saturday, March 15, 2014 8:59:09 AM UTC-3, Clinton Gormley wrote: > To add to what Binh said, you really shouldn't add field names like this: > On 14 March 2014 21:20, Pablo Musa \<[pabli...@gmail.com](mailto:pabli...@gmail.com) \\>wrote: > > > { > > "title":"The greatest band ever - Urban Legion", > > "greatest\_x" : 1, > > "band\_x" : 1, > > "ever\_x" : 1, > > "Urban\_x": 1, > > "Legion\_x" : 1, > > "greatest\_y" : , > > "band\_y" : , > > "ever\_y" : , > > "Urban\_y": , > > "Legion\_y" : > > } > > You end up with an explosion of fields, and each field has an inverted > index associated with it. Your cluster state will eventually become > enormous. Any change to the cluster state (eg adding a field, changing an > index, changes to nodes etc) results in the cluster state being copied to > every node in the cluster. If the state is very large you will experience a > significant slow down. > > Instead of: > > [{ custom\_foo: xxx }, { custom\_bar: yyy }] > > Use nested fields with eg > [ > { type: "custom\_foo", value: "xxx" }, > { type: "custom\_bar", value: "yyy" } > ] > > That way you have only two fields. > > clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/a5e3f166-7c40-4bdf-b1cc-4854abdb4595%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a5e3f166-7c40-4bdf-b1cc-4854abdb4595%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [March 17, 2014, 7:29pm UTC](https://discuss.elastic.co/t/creating-dynamic-fields-from-a-field/16383/5 "2014-03-17T19:29:21Z")
There is an REST API: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. If you are using Java, you can avoid the network roundtrip by creating the AnalysisService locally. For hints, see the test class: [https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/index/analysis/AnalysisModuleTests.java](https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/index/analysis/AnalysisModuleTests.java) -- Ivan On Mon, Mar 17, 2014 at 12:11 PM, [pablitomusa@gmail.com](mailto:pablitomusa@gmail.com) wrote: > Thank you very much for the hints 😉 > > > If it matters, you can call the \_analyze API to produce the tokens > > before you inject your fields. > > Is there an URL there I can call? Or only using the internal API? > > > Your cluster state will eventually become enormous. > > Yes, I saw it coming but was postponing in dev phase. Thanks for the > solution, it will help very much!! > > Thanks again guys! > > --Pablo > > On Saturday, March 15, 2014 8:59:09 AM UTC-3, Clinton Gormley wrote: > > > To add to what Binh said, you really shouldn't add field names like this: > > On 14 March 2014 21:20, Pablo Musa [pabli...@gmail.com](mailto:pabli...@gmail.com) wrote: > > > > > { > > > "title":"The greatest band ever - Urban Legion", > > > "greatest\_x" : 1, > > > "band\_x" : 1, > > > "ever\_x" : 1, > > > "Urban\_x": 1, > > > "Legion\_x" : 1, > > > "greatest\_y" : , > > > "band\_y" : , > > > "ever\_y" : , > > > "Urban\_y": , > > > "Legion\_y" : > > > } > > > > You end up with an explosion of fields, and each field has an inverted > > index associated with it. Your cluster state will eventually become > > enormous. Any change to the cluster state (eg adding a field, changing an > > index, changes to nodes etc) results in the cluster state being copied to > > every node in the cluster. If the state is very large you will experience a > > significant slow down. > > > > Instead of: > > > > [{ custom\_foo: xxx }, { custom\_bar: yyy }] > > > > Use nested fields with eg > > [ > > { type: "custom\_foo", value: "xxx" }, > > { type: "custom\_bar", value: "yyy" } > > ] > > > > That way you have only two fields. > > > > clint > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/a5e3f166-7c40-4bdf-b1cc-4854abdb4595%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a5e3f166-7c40-4bdf-b1cc-4854abdb4595%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/a5e3f166-7c40-4bdf-b1cc-4854abdb4595%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/a5e3f166-7c40-4bdf-b1cc-4854abdb4595%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAuVXV9Cu\_XY5sVuwU3J04iQcC%3Durw2tHnZRbdPG4WBxg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAuVXV9Cu_XY5sVuwU3J04iQcC%3Durw2tHnZRbdPG4WBxg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Pablo\_Musa](https://avatars.discourse-cdn.com/v4/letter/p/34f0e0/32.png) [@Pablo\_Musa](https://discuss.elastic.co/u/Pablo_Musa)\ **Post date:** [March 20, 2014, 7:06pm UTC](https://discuss.elastic.co/t/creating-dynamic-fields-from-a-field/16383/6 "2014-03-20T19:06:03Z")
Ok, I tried all the hints but now I have can't solve my original problem. I need to do an update to a value of type custom\_foo. In my previous approach I would do ctx.\_source.custom\_foo.value+=1. But now, there is a vector and I dont know which index is custom\_foo. Is there any fast method to get an nested object by value? Thanks, Pablo 2014-03-17 16:29 GMT-03:00 Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com): > There is an REST API: > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-analyze.html) > > If you are using Java, you can avoid the network roundtrip by creating the > AnalysisService locally. For hints, see the test class: > [https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/index/analysis/AnalysisModuleTests.java](https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/index/analysis/AnalysisModuleTests.java) > > -- > Ivan > > On Mon, Mar 17, 2014 at 12:11 PM, [pablitomusa@gmail.com](mailto:pablitomusa@gmail.com) wrote: > > > Thank you very much for the hints 😉 > > > > > If it matters, you can call the \_analyze API to produce the tokens > > > before you inject your fields. > > > > Is there an URL there I can call? Or only using the internal API? > > > > > Your cluster state will eventually become enormous. > > > > Yes, I saw it coming but was postponing in dev phase. Thanks for the > > solution, it will help very much!! > > > > Thanks again guys! > > > > --Pablo > > > > On Saturday, March 15, 2014 8:59:09 AM UTC-3, Clinton Gormley wrote: > > > > > To add to what Binh said, you really shouldn't add field names like this: > > > On 14 March 2014 21:20, Pablo Musa [pabli...@gmail.com](mailto:pabli...@gmail.com) wrote: > > > > > > > { > > > > "title":"The greatest band ever - Urban Legion", > > > > "greatest\_x" : 1, > > > > "band\_x" : 1, > > > > "ever\_x" : 1, > > > > "Urban\_x": 1, > > > > "Legion\_x" : 1, > > > > "greatest\_y" : , > > > > "band\_y" : , > > > > "ever\_y" : , > > > > "Urban\_y": , > > > > "Legion\_y" : > > > > } > > > > > > You end up with an explosion of fields, and each field has an inverted > > > index associated with it. Your cluster state will eventually become > > > enormous. Any change to the cluster state (eg adding a field, changing an > > > index, changes to nodes etc) results in the cluster state being copied to > > > every node in the cluster. If the state is very large you will experience a > > > significant slow down. > > > > > > Instead of: > > > > > > [{ custom\_foo: xxx }, { custom\_bar: yyy }] > > > > > > Use nested fields with eg > > > [ > > > { type: "custom\_foo", value: "xxx" }, > > > { type: "custom\_bar", value: "yyy" } > > > ] > > > > > > That way you have only two fields. > > > > > > clint > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/a5e3f166-7c40-4bdf-b1cc-4854abdb4595%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a5e3f166-7c40-4bdf-b1cc-4854abdb4595%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/a5e3f166-7c40-4bdf-b1cc-4854abdb4595%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/a5e3f166-7c40-4bdf-b1cc-4854abdb4595%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/TYLV9Leqfg8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/TYLV9Leqfg8/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAuVXV9Cu\_XY5sVuwU3J04iQcC%3Durw2tHnZRbdPG4WBxg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAuVXV9Cu_XY5sVuwU3J04iQcC%3Durw2tHnZRbdPG4WBxg%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAuVXV9Cu\_XY5sVuwU3J04iQcC%3Durw2tHnZRbdPG4WBxg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAuVXV9Cu_XY5sVuwU3J04iQcC%3Durw2tHnZRbdPG4WBxg%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAF6PhFJjiiTEKt-Y0vWOd9dVDQNtywbDiWE%3Do%3D\_hgTLRBbV%3DKg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAF6PhFJjiiTEKt-Y0vWOd9dVDQNtywbDiWE%3Do%3D_hgTLRBbV%3DKg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Pablo\_Musa](https://avatars.discourse-cdn.com/v4/letter/p/34f0e0/32.png) [@Pablo\_Musa](https://discuss.elastic.co/u/Pablo_Musa)\ **Post date:** [March 24, 2014, 11:25am UTC](https://discuss.elastic.co/t/creating-dynamic-fields-from-a-field/16383/7 "2014-03-24T11:25:08Z")
I solved my problem by using parent/child. Now I can use: must + should -\> has\_child -\> function\_score. It is working just fine. Just need to check performance. Thanks --Pablo On Thursday, March 20, 2014 4:06:03 PM UTC-3, Pablo Musa wrote: > Ok, I tried all the hints but now I have can't solve my original problem. > > I need to do an update to a value of type custom\_foo. > In my previous approach I would do ctx.\_source.custom\_foo.value+=1. > But now, there is a vector and I dont know which index is custom\_foo. > > Is there any fast method to get an nested object by value? > > Thanks, > Pablo > > 2014-03-17 16:29 GMT-03:00 Ivan Brusic: > > > There is an REST API: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-analyze.html) > > > > If you are using Java, you can avoid the network roundtrip by creating > > the AnalysisService locally. For hints, see the test class: > > [https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/index/analysis/AnalysisModuleTests.java](https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/index/analysis/AnalysisModuleTests.java) > > > > -- > > Ivan > > > > On Mon, Mar 17, 2014 at 12:11 PM,Pablo Musa wrote: > > > > > Thank you very much for the hints 😉 > > > > > > > If it matters, you can call the \_analyze API to produce the tokens > > > > before you inject your fields. > > > > > > Is there an URL there I can call? Or only using the internal API? > > > > > > > Your cluster state will eventually become enormous. > > > > > > Yes, I saw it coming but was postponing in dev phase. Thanks for the > > > solution, it will help very much!! > > > > > > Thanks again guys! > > > > > > --Pablo > > > > > > On Saturday, March 15, 2014 8:59:09 AM UTC-3, Clinton Gormley wrote: > > > > > > > To add to what Binh said, you really shouldn't add field names like > > > > this: > > > > On 14 March 2014 21:20, Pablo Musa wrote: > > > > > > > > > { > > > > > "title":"The greatest band ever - Urban Legion", > > > > > "greatest\_x" : 1, > > > > > "band\_x" : 1, > > > > > "ever\_x" : 1, > > > > > "Urban\_x": 1, > > > > > "Legion\_x" : 1, > > > > > "greatest\_y" : , > > > > > "band\_y" : , > > > > > "ever\_y" : , > > > > > "Urban\_y": , > > > > > "Legion\_y" : > > > > > } > > > > > > > > You end up with an explosion of fields, and each field has an inverted > > > > index associated with it. Your cluster state will eventually become > > > > enormous. Any change to the cluster state (eg adding a field, changing an > > > > index, changes to nodes etc) results in the cluster state being copied to > > > > every node in the cluster. If the state is very large you will experience a > > > > significant slow down. > > > > > > > > Instead of: > > > > > > > > [{ custom\_foo: xxx }, { custom\_bar: yyy }] > > > > > > > > Use nested fields with eg > > > > [ > > > > { type: "custom\_foo", value: "xxx" }, > > > > { type: "custom\_bar", value: "yyy" } > > > > ] > > > > > > > > That way you have only two fields. > > > > > > > > clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/eb1ae45b-f84c-4658-88f8-52e3d539db2c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/eb1ae45b-f84c-4658-88f8-52e3d539db2c%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:41am UTC](https://discuss.elastic.co/t/creating-dynamic-fields-from-a-field/16383/8 "2017-07-06T01:41:11Z")