# Does it make sense to separate rarely used documents for faster performance?

**URL:** <https://discuss.elastic.co/t/does-it-make-sense-to-separate-rarely-used-documents-for-faster-performance/282550>\
**Category:** Elasticsearch\
**Created:** [August 26, 2021, 8:45am UTC](https://discuss.elastic.co/t/does-it-make-sense-to-separate-rarely-used-documents-for-faster-performance/282550 "2021-08-26T08:45:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yoan\_Arnaudov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yoan_arnaudov/32/93740_2.png) [@Yoan\_Arnaudov](https://discuss.elastic.co/u/Yoan_Arnaudov)\
**Post date:** [August 26, 2021, 8:45am UTC](https://discuss.elastic.co/t/does-it-make-sense-to-separate-rarely-used-documents-for-faster-performance/282550/1 "2021-08-26T08:45:32Z")

</div>

I'm designing a system which will have **~2M** documents and **95%** percent of the queries will be targeted against **40-50k** documents.

Documents in the system have two states bool fields `active` and `inactive`.

I have ~50k in the state `active` and the rest 1.95 million are `inactive`.

So my questions is does it make sense to separate these documents? How can I separate them? Do I need to worry about this at all? Maybe I can use shards and a hash function which chooses buckets depending on the state active/inactive?

I can also introduce two indexes, one which holds `active` and one for the `inactive`, that way most of the queries will do a lookup in small database. This solution however will require more maintenance and can cause more headaches in the long run.

I'm not fully aware of the capabilities provided by ES and I would love to hear someone with more experience.

> <https://stackoverflow.com/questions/68935253/elasticsearch-does-it-make-sense-to-separate-rarely-used-documents-for-faster>

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [August 26, 2021, 3:34pm UTC](https://discuss.elastic.co/t/does-it-make-sense-to-separate-rarely-used-documents-for-faster-performance/282550/2 "2021-08-26T15:34:38Z")

</div>

Two million documents is not that much, so I would go with a single index to start simple, and just have an `active` boolean for each document. Make sure you use filters, so that active/inactive documents can make use of the node query cache.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 23, 2021, 3:35pm UTC](https://discuss.elastic.co/t/does-it-make-sense-to-separate-rarely-used-documents-for-faster-performance/282550/3 "2021-09-23T15:35:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.