# Recommendations to organize index with Master-detail data to be queried with lots of aggregations?

**URL:** <https://discuss.elastic.co/t/recommendations-to-organize-index-with-master-detail-data-to-be-queried-with-lots-of-aggregations/33195>\
**Category:** Elasticsearch\
**Created:** [October 28, 2015, 10:25pm UTC](https://discuss.elastic.co/t/recommendations-to-organize-index-with-master-detail-data-to-be-queried-with-lots-of-aggregations/33195 "2015-10-28T22:25:53Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Xavi\_Ametller](https://avatars.discourse-cdn.com/v4/letter/x/b5ac83/32.png) [@Xavi\_Ametller](https://discuss.elastic.co/u/Xavi_Ametller)\
**Post date:** [October 28, 2015, 10:25pm UTC](https://discuss.elastic.co/t/recommendations-to-organize-index-with-master-detail-data-to-be-queried-with-lots-of-aggregations/33195/1 "2015-10-28T22:25:53Z")

</div>

Hi everyone,

_Disclosure_

- _I'm new to ES (one week) so feel free to correct me anywhere._
- _I am running a proof of concept and have done zero perfomance optimization, I want to check if I'm on the right track before performance tunning._

My use case is as follows:

1. Set of data which can be categorized as master-detail. There are around 4K masters with 10M details, so each master has on average 2500 details.
2. I want to create a "search" page displaying possible facets to the user. For each facet I want to display the **amount of Master elements** which have refinements .
3. I may have between 20-30 facets
4. I want it fast 😄

I toyed around with ES and started plain simple. I just create one document in ES foreach detail (so 10M documents in total). On each document I include both the master and the detail information (so yes, I repeat a lot of information 2500 times).  
With this structure, I can create a query like this:

```
GET /{index}/{type}/_search
{    
  "size": { ... },
  "sort": { ... },
  "query": { ... },
  "aggs" {
    # This structure is repeated 20-30 times, once for each facet I want to allow filtering by
    "first_filter": {
        "terms": {
           "field": "first_filter_field"
         },
       "aggs": {
          "distinct_master_field_id": {
             "cardinality": {
                "field": "master_field_id"
        },
   "second_filter": {
        "terms": {
           "field": "second_filter_field"
         },
       "aggs": {
          "distinct_master_field_id": {
             "cardinality": {
                "field": "master_field_id"
        }
      }
      ....
    }
  }
}

```

Thing is, I'm far from the performance I would like to have, it takes seconds and I was heading for millis. My question is: Is this the best/good way to organize and query this information with such a use case?

Considerations:

- [Parent child relationships][1] - This data structure seems to match my scenario, but I am not worried about modifying the index and am really concerned about performance. According to documentation performance is _worse_ with parent-child structures
- Am I abusing cardinality sub-aggregates? I'm always applying the same cardinality sub-aggregate... maybe there's a better way to organize the query
- The option to grow horizontally is always there, don't want to go there from the very beginning (altough I may have to...)

Thank you all for your time. Any feedback will be appreciated!  
Xavi  
[1]: [https://www.elastic.co/guide/en/elasticsearch/guide/current/parent-child.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/parent-child.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:41pm UTC](https://discuss.elastic.co/t/recommendations-to-organize-index-with-master-detail-data-to-be-queried-with-lots-of-aggregations/33195/2 "2017-07-05T23:41:58Z")

</div>