# One index by dataset?

**URL:** <https://discuss.elastic.co/t/one-index-by-dataset/324684>\
**Category:** Elasticsearch\
**Created:** [February 3, 2023, 7:07pm UTC](https://discuss.elastic.co/t/one-index-by-dataset/324684 "2023-02-03T19:07:03Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![m4rk](https://avatars.discourse-cdn.com/v4/letter/m/edb3f5/32.png) [@m4rk](https://discuss.elastic.co/u/m4rk)\
**Post date:** [February 3, 2023, 7:07pm UTC](https://discuss.elastic.co/t/one-index-by-dataset/324684/1 "2023-02-03T19:07:03Z")

</div>

I need to store multiple datasets in Elasticsearch, each containing a high number of documents (something between 100,000 and 1,000,000). They have essentially the same structure. Since I use /analyze only one dataset at a time, I think it would be a good choice to create an index for each of them. What do you think?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [February 3, 2023, 7:23pm UTC](https://discuss.elastic.co/t/one-index-by-dataset/324684/2 "2023-02-03T19:23:03Z")

</div>

How many dataset do you expect to have? What is the average document size?

---

<div class="post-metadata">

**Author:** ![m4rk](https://avatars.discourse-cdn.com/v4/letter/m/edb3f5/32.png) [@m4rk](https://discuss.elastic.co/u/m4rk)\
**Post date:** [February 3, 2023, 7:27pm UTC](https://discuss.elastic.co/t/one-index-by-dataset/324684/3 "2023-02-03T19:27:08Z")

</div>

Probably something like 20 datasets.  
Each dataset could be 1 Go to 15 Go.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [February 3, 2023, 7:29pm UTC](https://discuss.elastic.co/t/one-index-by-dataset/324684/4 "2023-02-03T19:29:21Z")

</div>

Then an index per dataset sounds reasonable.

---

<div class="post-metadata">

**Author:** ![m4rk](https://avatars.discourse-cdn.com/v4/letter/m/edb3f5/32.png) [@m4rk](https://discuss.elastic.co/u/m4rk)\
**Post date:** [February 3, 2023, 7:31pm UTC](https://discuss.elastic.co/t/one-index-by-dataset/324684/5 "2023-02-03T19:31:22Z")

</div>

Excellent, thank you!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 3, 2023, 7:31pm UTC](https://discuss.elastic.co/t/one-index-by-dataset/324684/6 "2023-03-03T19:31:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.