# Dynamically adding new fields from a root mapper **URL:** \ **Category:** Elasticsearch\ **Created:** [August 21, 2014, 10:47am UTC](https://discuss.elastic.co/t/dynamically-adding-new-fields-from-a-root-mapper/19389 "2014-08-21T10:47:10Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Jakub\_Kotowski](https://avatars.discourse-cdn.com/v4/letter/j/7ba0ec/32.png) [@Jakub\_Kotowski](https://discuss.elastic.co/u/Jakub_Kotowski)\ **Post date:** [August 21, 2014, 10:47am UTC](https://discuss.elastic.co/t/dynamically-adding-new-fields-from-a-root-mapper/19389/1 "2014-08-21T10:47:10Z")
Hi, I am creating a plugin that analyzes document being indexed and based on the analysis it adds new fields to it. It is done from a root mapper and it works similarly as the attachment mapper. I need to use the root mapper because I need the whole document for analysis, not individual fields. I also don't have a fixed predefined set of fields that can be generated. I'd like to ask what is the correct way of updating the mapping in this scenario. If I don't update the mapping then I'm getting a NPE: Caused by: java.lang.NullPointerException org.elasticsearch.index.fieldvisitor.FieldsVisitor.postProcess(FieldsVisitor.java:70) org.elasticsearch.index.get.ShardGetService.innerGetLoadFromStoredFields(ShardGetService.java:333) org.elasticsearch.index.get.ShardGetService.innerGet(ShardGetService.java:212) org.elasticsearch.index.get.ShardGetService.get(ShardGetService.java:106) org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:109) org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:43) That's because there is no FieldMapper for the newly added field. I can add the field mapper this way: context.docMapper().addFieldMappers(ImmutableSet.of(field.mapper)) and then it seems to work. No NPE, no warnings in the log. However, to create the field mapper, I need a Mapper.BuilderContext, so that I can do e.g. field.mapper = MapperBuilders.stringField(field.name()).build(builderContext)); This seems to be a problem because the BuilderContext is available only from my root mapper's Builder. I pass the BuilderContext to my root mapper which then uses it to create the field mappers. But this seems hacky/wrong. What would be the correct approach? Will the mapping be otherwise updated somehow by Elasticsearch in this case? Or how exactly should I go about implementing my root mapper's toXContent() method? You can see the mapper here: [https://github.com/sindicetech/smartindexing/blob/issue-1/cogito-elasticsearch/src/main/java/net/expertsystem/connector/elasticsearch/index/CogitoMapper.java](https://github.com/sindicetech/smartindexing/blob/issue-1/cogito-elasticsearch/src/main/java/net/expertsystem/connector/elasticsearch/index/CogitoMapper.java) (work in progress/playground). Thanks, Jakub -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c3722146-2929-4a3d-b82e-07a54ddef6d0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c3722146-2929-4a3d-b82e-07a54ddef6d0%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Jakub\_Kotowski](https://avatars.discourse-cdn.com/v4/letter/j/7ba0ec/32.png) [@Jakub\_Kotowski](https://discuss.elastic.co/u/Jakub_Kotowski)\ **Post date:** [August 22, 2014, 12:07pm UTC](https://discuss.elastic.co/t/dynamically-adding-new-fields-from-a-root-mapper/19389/2 "2014-08-22T12:07:40Z")
Hi, I'm wondering if anyone could provide some hints... 🙂 Pointers where to learn more about Mappers, FieldMappers, RootMappers, adding new fields, how mapping is updated and its relation to toXContent() of mappers would be appreciated too. Also paths vs. full paths vs. multi\_field / fields, etc. Cheers, Jakub On Thursday, August 21, 2014 12:47:10 PM UTC+2, Jakub Kotowski wrote: > Hi, > > I am creating a plugin that analyzes document being indexed and based on > the analysis it adds new fields to it. It is done from a root mapper and it > works similarly as the attachment mapper. > > I need to use the root mapper because I need the whole document for > analysis, not individual fields. I also don't have a fixed predefined set > of fields that can be generated. > > I'd like to ask what is the correct way of updating the mapping in this > scenario. > > If I don't update the mapping then I'm getting a NPE: > > Caused by: java.lang.NullPointerException > > org.elasticsearch.index.fieldvisitor.FieldsVisitor.postProcess(FieldsVisitor.java:70) > > org.elasticsearch.index.get.ShardGetService.innerGetLoadFromStoredFields(ShardGetService.java:333) > > org.elasticsearch.index.get.ShardGetService.innerGet(ShardGetService.java:212) > > org.elasticsearch.index.get.ShardGetService.get(ShardGetService.java:106) > > org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:109) > > org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:43) > > That's because there is no FieldMapper for the newly added field. > > I can add the field mapper this > way: context.docMapper().addFieldMappers(ImmutableSet.of(field.mapper)) and > then it seems to work. No NPE, no warnings in the log. > > However, to create the field mapper, I need a Mapper.BuilderContext, so > that I can do e.g. field.mapper = MapperBuilders.stringField(field.name > ()).build(builderContext)); > > This seems to be a problem because the BuilderContext is available only > from my root mapper's Builder. I pass the BuilderContext to my root mapper > which then uses it to create the field mappers. But this seems hacky/wrong. > > What would be the correct approach? Will the mapping be otherwise updated > somehow by Elasticsearch in this case? Or how exactly should I go about > implementing my root mapper's toXContent() method? > > You can see the mapper here: > [https://github.com/sindicetech/smartindexing/blob/issue-1/cogito-elasticsearch/src/main/java/net/expertsystem/connector/elasticsearch/index/CogitoMapper.java](https://github.com/sindicetech/smartindexing/blob/issue-1/cogito-elasticsearch/src/main/java/net/expertsystem/connector/elasticsearch/index/CogitoMapper.java) > (work in progress/playground). > > Thanks, > > Jakub -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/48f1f2a5-95d2-4182-833a-135798e3f44a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/48f1f2a5-95d2-4182-833a-135798e3f44a%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:07am UTC](https://discuss.elastic.co/t/dynamically-adding-new-fields-from-a-root-mapper/19389/3 "2017-07-06T01:07:01Z")
Closed on July 6, 2017, 1:07am UTC