# Storing documents with some arbitrary data

**URL:** <https://discuss.elastic.co/t/storing-documents-with-some-arbitrary-data/52075>\
**Category:** Elasticsearch\
**Created:** [June 7, 2016, 1:13pm UTC](https://discuss.elastic.co/t/storing-documents-with-some-arbitrary-data/52075 "2016-06-07T13:13:26Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jameso](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jameso/32/10182_2.png) [@jameso](https://discuss.elastic.co/u/jameso)\
**Post date:** [June 7, 2016, 1:13pm UTC](https://discuss.elastic.co/t/storing-documents-with-some-arbitrary-data/52075/1 "2016-06-07T13:13:26Z")

</div>

We need to store documents for our users. Each user will have a number of documents that all have the same core fields, we then allow users to add their own arbitrary fields to their documents.

Reading this article: [https://www.elastic.co/guide/en/elasticsearch/guide/current/mapping.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/mapping.html). It says _"Types are not as well suited for entirely different types of data."_ These documents won't all be entirely different types, does that mean it shouldn't be an issue?

Is there an accepted best practice when storing data that doesn't all have exactly the same format?

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [June 7, 2016, 3:09pm UTC](https://discuss.elastic.co/t/storing-documents-with-some-arbitrary-data/52075/2 "2016-06-07T15:09:16Z")

</div>

Yeah, dynamic:false. That'll let you manually configure all the fields that  
you want indexed and the other fields are just stored in the source. They  
can be fetched but that is it.

Letting users create whatever fields they want is problematic for two  
reason:

1. Type clashes. You can resolve that by naming fields after their type so  
they done clash. Like phone\_number\_string. Sure, it is lame but you are  
working around inconsistent user data.
2. Too many fields and it's corallary, sparse fields. This is trouble. Each  
field has an overhead in the index and in the mapping. Adding a field is a  
cluster state change event too. Sparse fields don't store well either in  
doc values.

Both of these things you can work around but they are certainly "advanced"  
use cases.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:45pm UTC](https://discuss.elastic.co/t/storing-documents-with-some-arbitrary-data/52075/3 "2017-07-05T22:45:34Z")

</div>