# Is it worse to have very large documents/indices with a ton of data, or split that data into smaller documents/indices using a relational paradigm?

**URL:** <https://discuss.elastic.co/t/is-it-worse-to-have-very-large-documents-indices-with-a-ton-of-data-or-split-that-data-into-smaller-documents-indices-using-a-relational-paradigm/293284>\
**Category:** Elasticsearch\
**Created:** [January 2, 2022, 3:23am UTC](https://discuss.elastic.co/t/is-it-worse-to-have-very-large-documents-indices-with-a-ton-of-data-or-split-that-data-into-smaller-documents-indices-using-a-relational-paradigm/293284 "2022-01-02T03:23:34Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![annmarie-switzer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/annmarie-switzer/32/82870_2.png) [@annmarie-switzer](https://discuss.elastic.co/u/annmarie-switzer)\
**Post date:** [January 2, 2022, 3:23am UTC](https://discuss.elastic.co/t/is-it-worse-to-have-very-large-documents-indices-with-a-ton-of-data-or-split-that-data-into-smaller-documents-indices-using-a-relational-paradigm/293284/1 "2022-01-02T03:23:34Z")

</div>

Say I have a store and each product I sell has a document in Elasticsearch:

```auto
{
    id: "abc",
    name: "mittens",
    price: "$10.00"
},
{
    id: "def",
    name: "hat",
    price: "$13.00"
}

```

My store's website allows users to log in and review products. However, at this store, the reviews shown are on a user-by-user basis rather than aggregated. Meaning, if Harry gives `mittens` a 4-star review, and Hermione gives them a 2-star review, I want Harry to see 4 stars and Hermione to see 2.

In order to achieve this, I need to somewhere store each individual review for every user. I'm wondering what the most performant and efficient way is to store this data.

**Option 1:** I can store it on the product document itself, like so:

```auto
{
    id: "abc",
    name: "mittens",
    price: "$10.00",
    ratings: [
        { user: "Harry", stars: 4 },
        { user: "Hermione", stars: 2 },
        ...
    ]
}

```

This makes retrieving and displaying the data really easy, but I wonder how reasonable this model is if I had users in the thousands.

**Option 2:** use separate indices in a relational-like model. The product index would have documents with product data only, no ratings. The second index would be just for users' ratings and would have documents like so:

```auto
{
    user: "Harry",
    ratings: [
        { id: "abc", stars: 4 },
        ...
    ]
},
{
    user: "Hermione",
    ratings: [
        { id: "abc", stars: 2 },
        { id: "def", stars: 3 },
        ...
    ]
}

```

This option keeps the individual documents in each index relatively small, but it complicates querying and displaying data. I'm also concerned that using this relational-paradigm is "wrong" for something like Elasticsearch.

So at a high level, I'm wondering which option is the most performant and scalable. Or if there is perhaps an even better option that I haven't thought of. Thanks!

---

<div class="post-metadata">

**Author:** ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)\
**Post date:** [January 3, 2022, 3:59pm UTC](https://discuss.elastic.co/t/is-it-worse-to-have-very-large-documents-indices-with-a-ton-of-data-or-split-that-data-into-smaller-documents-indices-using-a-relational-paradigm/293284/2 "2022-01-03T15:59:23Z")

</div>

It depends on scale, review frequency and performance requirement.  
If the number of users and products are some thousands, any options may be acceptable.

both option 1 and 2 have the same updating problem for scaling. To update documents in Elasticsearch, documents were deleted and new comlete documents were indexed internally.  
One possible option is similar to option 2 but index each review as one document and use aggregation on query.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 31, 2022, 3:59pm UTC](https://discuss.elastic.co/t/is-it-worse-to-have-very-large-documents-indices-with-a-ton-of-data-or-split-that-data-into-smaller-documents-indices-using-a-relational-paradigm/293284/3 "2022-01-31T15:59:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.