# Dynamic schema from NoSQL river **URL:** \ **Category:** Elasticsearch\ **Created:** [February 1, 2013, 4:40am UTC](https://discuss.elastic.co/t/dynamic-schema-from-nosql-river/10582 "2013-02-01T04:40:13Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![nwood888](https://avatars.discourse-cdn.com/v4/letter/n/71c47a/32.png) [@nwood888](https://discuss.elastic.co/u/nwood888)\ **Post date:** [February 1, 2013, 4:40am UTC](https://discuss.elastic.co/t/dynamic-schema-from-nosql-river/10582/1 "2013-02-01T04:40:13Z")
We're exploring using a NoSQL system (CouchDB, Couchbase, etc) as our primary data store and ES with a river plugin for search in a multitenant system where each user can define their own set of custom fields. NoSQL does well with this, but I'm concerned about ES indexing these custom fields. We might have user\_1 create a custom field called "field1" that stores a date data type, and user\_2 create a field with the same name that stores an integer data type. Would it make sense to force the data type to be part of the field name? Like date\_field1, etc? It seems like ES might have an easier time with this. As a second concern - will ES have issues with the # of unique field names? Over time the index could have thousands of unique field names if we give our users the option of defining their own field names. Creating a separate index per user will not work considering the scale we're after. Nick -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 1, 2013, 7:49am UTC](https://discuss.elastic.co/t/dynamic-schema-from-nosql-river/10582/2 "2013-02-01T07:49:05Z")
Instead of thinking creating an index per user, I would think about creating a type per user. Each type has its own mapping so you can mix things. Using also templates, you can apply a common pattern mapping to each new user and let the user add its own new field. Does it help? Le 1 févr. 2013 à 05:40, Nick Wood [nwood888@gmail.com](mailto:nwood888@gmail.com) a écrit : > We're exploring using a NoSQL system (CouchDB, Couchbase, etc) as our primary data store and ES with a river plugin for search in a multitenant system where each user can define their own set of custom fields. NoSQL does well with this, but I'm concerned about ES indexing these custom fields. > > We might have user\_1 create a custom field called "field1" that stores a date data type, and user\_2 create a field with the same name that stores an integer data type. > > Would it make sense to force the data type to be part of the field name? Like date\_field1, etc? It seems like ES might have an easier time with this. > > As a second concern - will ES have issues with the # of unique field names? Over time the index could have thousands of unique field names if we give our users the option of defining their own field names. > > Creating a separate index per user will not work considering the scale we're after. > > Nick > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![nwood888](https://avatars.discourse-cdn.com/v4/letter/n/71c47a/32.png) [@nwood888](https://discuss.elastic.co/u/nwood888)\ **Post date:** [February 7, 2013, 6:19am UTC](https://discuss.elastic.co/t/dynamic-schema-from-nosql-river/10582/3 "2013-02-07T06:19:19Z")
Thank you David. From a performance standpoint, it makes sense that we wouldn't want to create an index per user. Will adding a type per user cause any issues if there are, say 1 million different types? "Each type has it's own mapping so you can mix things" - Are you sure this is the case? I remember reading somewhere that even between types, fields with the same name that are different types can cause problems. I can't for the life of me find the post that said this, so maybe I was dreaming 🙂 Two more somewhat related questions: 1. Is there a way to specify the type without requiring that it be the "root" element in the json object? I ask because it would be cleaner and easier for us to put the type in a regular field if we can somehow define it's location through mapping. 2. When you refer to using templates, are you talking about the dynamic\_templates section of this page - [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html)? Is there a better place to learn about templates because I read that and it's still not obvious how to use them. Thanks again for your help! Nick On Fri, Feb 1, 2013 at 12:49 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Instead of thinking creating an index per user, I would think about > creating a type per user. > Each type has its own mapping so you can mix things. > > Using also templates, you can apply a common pattern mapping to each new > user and let the user add its own new field. > > Does it help? > > Le 1 févr. 2013 à 05:40, Nick Wood [nwood888@gmail.com](mailto:nwood888@gmail.com) a écrit : > > We're exploring using a NoSQL system (CouchDB, Couchbase, etc) as our > primary data store and ES with a river plugin for search in a multitenant > system where each user can define their own set of custom fields. NoSQL > does well with this, but I'm concerned about ES indexing these custom > fields. > > We might have user\_1 create a custom field called "field1" that stores a > date data type, and user\_2 create a field with the same name that stores an > integer data type. > > Would it make sense to force the data type to be part of the field name? > Like date\_field1, etc? It seems like ES might have an easier time with > this. > > As a second concern - will ES have issues with the # of unique field > names? Over time the index could have thousands of unique field names if > we give our users the option of defining their own field names. > > Creating a separate index per user will not work considering the scale > we're after. > > Nick > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:52am UTC](https://discuss.elastic.co/t/dynamic-schema-from-nosql-river/10582/4 "2017-07-06T02:52:39Z")