# Best practices documents with large field count with an eye to performance

**URL:** <https://discuss.elastic.co/t/best-practices-documents-with-large-field-count-with-an-eye-to-performance/180817>\
**Category:** Elasticsearch\
**Created:** [May 13, 2019, 1:00pm UTC](https://discuss.elastic.co/t/best-practices-documents-with-large-field-count-with-an-eye-to-performance/180817 "2019-05-13T13:00:53Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Niels\_van\_Rijn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niels_van_rijn/32/45639_2.png) [@Niels\_van\_Rijn](https://discuss.elastic.co/u/Niels_van_Rijn)\
**Post date:** [May 13, 2019, 1:00pm UTC](https://discuss.elastic.co/t/best-practices-documents-with-large-field-count-with-an-eye-to-performance/180817/1 "2019-05-13T13:00:53Z")

</div>

Hi! Im pretty new to the whole ELK stack.

My data consists of around 10.600 json documents, each document having between 1000 - 50.000 properties/fields([here](https://pastebin.com/JUbY1pL3) is an example of a small document).  
A large amount of the fields needs to be searchable (all the "\*#text" and "\*al\*" fields).

Because the documents have different structures/types(and types are being removed) I will use 1 index per document as suggested [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html#_index_per_document_type)

But I'm running into the issue that I have to many shards open per cluster.  
Should I use less shards per index/document (currently 5 shards per index)?  
Or should I add more clusters?

I feel like I'm missing something, my data is not that large (in gb's). I see examples of 50gb. My complete dataset is only around 2gb but performance with 50.000 fields is very poor from my initial testing.

How would one store this kind of dataset?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 10, 2019, 1:00pm UTC](https://discuss.elastic.co/t/best-practices-documents-with-large-field-count-with-an-eye-to-performance/180817/2 "2019-06-10T13:00:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.