# Dynamic data (no code) scenario strategy

**URL:** <https://discuss.elastic.co/t/dynamic-data-no-code-scenario-strategy/334870>\
**Category:** Elasticsearch\
**Created:** [May 31, 2023, 7:38pm UTC](https://discuss.elastic.co/t/dynamic-data-no-code-scenario-strategy/334870 "2023-05-31T19:38:44Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Zak\_Sesti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zak_sesti/32/121689_2.png) [@Zak\_Sesti](https://discuss.elastic.co/u/Zak_Sesti)\
**Post date:** [May 31, 2023, 7:38pm UTC](https://discuss.elastic.co/t/dynamic-data-no-code-scenario-strategy/334870/1 "2023-05-31T19:38:44Z")

</div>

I use ES for searching of my basic CRUD constructs. But now we need to expand to help us search, sort, paginate our no-code constructs. These are json documents that have 100% dynamic fields. Some rough numbers:

- We have thousands of organizations
- Each organization has an average of 20 groups but some have several 100's of groups
- Each group has a single no-code data definition. The documents for each group will share the same attributes but all other documents in other groups are guaranteed not to share any of the same attributes.
- Documents are small, average 252 bytes up to 14 kb
- The number of attributes in each document could be from 1-100 but averages around 20

The question is how can I index the no-code data? I can't throw them all in a single index obviously as it would have an enormous number of attributes.

Can I create an index per group?

That would result in 1000 orgs X 20 groups = 20000 indices and growing rapidly.

Is that within reason or far-fetched? What if it's 10x orgs or 100x orgs? Could it scale? Basically many small indices. I'm guessing not well.

My instinct is that we should map all dynamic no-code data down to static fields. Since we understand the data types we could map the data to something like:

`{ textField1: string, textField2: string, textFieldN: string, dateField1: Date, dateFieldX:Date etc..}`

We are willing to limit the number of attributes that can be mapped into ES so this approach seems doable but just clunky.

Is this the only option?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 6, 2023, 1:38am UTC](https://discuss.elastic.co/t/dynamic-data-no-code-scenario-strategy/334870/2 "2023-06-06T01:38:52Z")

</div>

Welcome to our community! 😃

> [@Zak\_Sesti](#):
>
> Is this the only option?

It's the only sensible option, yes.

The others are the ones you pointed out, specific indices, which don't really scale.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 4, 2023, 1:39am UTC](https://discuss.elastic.co/t/dynamic-data-no-code-scenario-strategy/334870/3 "2023-07-04T01:39:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.