# Elastic Query unexcepted aggregations doc\_count

**URL:** <https://discuss.elastic.co/t/elastic-query-unexcepted-aggregations-doc-count/356482>\
**Category:** Elasticsearch\
**Created:** [March 30, 2024, 8:42am UTC](https://discuss.elastic.co/t/elastic-query-unexcepted-aggregations-doc-count/356482 "2024-03-30T08:42:02Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vivek\_Burman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vivek_burman/32/118310_2.png) [@Vivek\_Burman](https://discuss.elastic.co/u/Vivek_Burman)\
**Post date:** [March 30, 2024, 8:42am UTC](https://discuss.elastic.co/t/elastic-query-unexcepted-aggregations-doc-count/356482/1 "2024-03-30T08:42:02Z")

</div>

Question: If you look at the ES Response the hits count is 22 but the aggregations doc\_count is 663. I didn't understand the reason behind it.

I was expecting that each doc from the hit will participate in one of the bucket aggs.

ES Query:

```auto
{
    "query": {
        "bool": {
            "filter": [
                {
                    "terms": {
                        "test_info.entity_type": [
                            "AUTO"
                        ]
                    }
                },
                {
                    "nested": {
                        "query": {
                            "nested": {
                                "query": {
                                    "bool": {
                                        "filter": [
                                            {
                                                "terms": {
                                                    "test_case.custom_field.pid": [
                                                        6398
                                                    ]
                                                }
                                            },
                                            {
                                                "terms": {
                                                    "test_case.custom_field.text_value": [
                                                        "Krit Shah",
                                                        "amar s",
                                                        "Vivek Burmanv6",
                                                        "vivek burman",
                                                        "vivek b"
                                                    ]
                                                }
                                            }
                                        ]
                                    }
                                },
                                "path": "test_case.custom_field"
                            }
                        },
                        "path": "test_case"
                    }
                },
                {
                    "term": {
                        "is_deleted": 0
                    }
                },
                {
                    "term": {
                        "project.is_deleted": 0
                    }
                },
                {
                    "terms": {
                        "project.pid": [
                            1969
                        ]
                    }
                }
            ]
        }
    },
    "size": 0,
    "aggs": {
        "aq_agg_test_case": {
            "nested": {
                "path": "test_case"
            },
            "aggs": {
                "aq_agg_custom_field": {
                    "nested": {
                        "path": "test_case.custom_field"
                    },
                    "aggs": {
                        "aq_filter_agg_unique_name": {
                            "terms": {
                                "field": "test_case.custom_field.unique_name",
                                "include": [
                                    "CUserList"
                                ]
                            },
                            "aggs": {
                                "aq_agg_text_value": {
                                    "terms": {
                                        "field": "test_case.custom_field.text_value"
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

```

ES Response:

```auto
{
    "_shards": {
        "failed": 0,
        "skipped": 0,
        "successful": 1,
        "total": 1
    },
    "aggregations": {
        "aq_agg_test_case": {
            "aq_agg_custom_field": {
                "aq_filter_agg_unique_name": {
                    "buckets": [
                        {
                            "aq_agg_text_value": {
                                "buckets": [
                                    {
                                        "doc_count": 448,
                                        "key": "Lakshmi N"
                                    },
                                    {
                                        "doc_count": 165,
                                        "key": "Vivek Burmanv6"
                                    },
                                    {
                                        "doc_count": 50,
                                        "key": "Krit Shah"
                                    }
                                ],
                                "doc_count_error_upper_bound": 0,
                                "sum_other_doc_count": 0
                            },
                            "doc_count": 663,
                            "key": "CUserList"
                        }
                    ],
                    "doc_count_error_upper_bound": 0,
                    "sum_other_doc_count": 0
                },
                "doc_count": 3978
            },
            "doc_count": 663
        }
    },
    "hits": {
        "hits": [],
        "max_score": null,
        "total": {
            "relation": "eq",
            "value": 22
        }
    },
    "timed_out": false,
    "took": 9
}

```

ES Index mapping:

```auto
{
    "poc_testresult_v1": {
        "mappings": {
            "dynamic": "false",
            "properties": {
                "_meta": {
                    "properties": {
                        "conf_dataset": {
                            "properties": {
                                "pid": {
                                    "type": "integer"
                                }
                            }
                        },
                        "conf_entity_metadata": {
                            "properties": {
                                "pid": {
                                    "type": "integer"
                                }
                            }
                        },
                        "conf_entity_status": {
                            "properties": {
                                "pid": {
                                    "type": "integer"
                                }
                            }
                        },
                        "conf_metadata": {
                            "properties": {
                                "pid": {
                                    "type": "integer"
                                }
                            }
                        },
                        "ext_int_work_item": {
                            "properties": {
                                "pid": {
                                    "type": "integer"
                                }
                            }
                        },
                        "metadata_value": {
                            "properties": {
                                "pid": {
                                    "type": "integer"
                                }
                            }
                        },
                        "project": {
                            "properties": {
                                "pid": {
                                    "type": "integer"
                                }
                            }
                        },
                        "scenario": {
                            "properties": {
                                "pid": {
                                    "type": "integer"
                                }
                            }
                        },
                        "scn_data_table_row": {
                            "properties": {
                                "pid": {
                                    "type": "integer"
                                }
                            }
                        },
                        "test_job_reln_defect_item": {
                            "properties": {
                                "pid": {
                                    "type": "integer"
                                }
                            }
                        },
                        "users": {
                            "properties": {
                                "pid": {
                                    "type": "integer"
                                }
                            }
                        }
                    }
                },
                "app_environment": {
                    "properties": {
                        "name": {
                            "type": "keyword"
                        },
                        "pid": {
                            "type": "integer"
                        }
                    }
                },
                "blocked": {
                    "type": "integer"
                },
                "browser_test": {
                    "type": "nested",
                    "properties": {
                        "browser": {
                            "type": "keyword"
                        },
                        "browser_version": {
                            "type": "keyword"
                        },
                        "os": {
                            "type": "keyword"
                        },
                        "os_version": {
                            "type": "keyword"
                        }
                    }
                },
                "completed_timestamp": {
                    "type": "date"
                },
                "deferred": {
                    "type": "integer"
                },
                "execution_param": {
                    "type": "nested",
                    "properties": {
                        "name": {
                            "type": "keyword"
                        },
                        "value": {
                            "type": "keyword"
                        }
                    }
                },
                "failed": {
                    "type": "integer"
                },
                "is_completed": {
                    "type": "integer"
                },
                "is_deleted": {
                    "type": "integer"
                },
                "job_type": {
                    "type": "keyword"
                },
                "mobile_app_test": {
                    "type": "nested",
                    "properties": {
                        "name": {
                            "type": "keyword"
                        },
                        "os": {
                            "type": "keyword"
                        },
                        "os_version": {
                            "type": "keyword"
                        },
                        "udid": {
                            "type": "keyword"
                        }
                    }
                },
                "not_completed": {
                    "type": "integer"
                },
                "not_run": {
                    "type": "integer"
                },
                "passed": {
                    "type": "integer"
                },
                "pid": {
                    "type": "integer"
                },
                "project": {
                    "properties": {
                        "deleted_timestamp": {
                            "type": "date"
                        },
                        "is_deleted": {
                            "type": "integer"
                        },
                        "pid": {
                            "type": "integer"
                        },
                        "project_display_name": {
                            "type": "keyword"
                        },
                        "project_name": {
                            "type": "keyword",
                            "index": false
                        }
                    }
                },
                "purpose": {
                    "type": "text"
                },
                "run_status": {
                    "type": "keyword"
                },
                "skipped": {
                    "type": "integer"
                },
                "start_timestamp": {
                    "type": "date"
                },
                "test_case": {
                    "type": "nested",
                    "properties": {
                        "custom_field": {
                            "type": "nested",
                            "properties": {
                                "bool_value": {
                                    "type": "integer"
                                },
                                "conf_entity_metadata_pid": {
                                    "type": "integer"
                                },
                                "date_value": {
                                    "type": "date"
                                },
                                "db_column_name": {
                                    "type": "keyword",
                                    "index": false
                                },
                                "is_required": {
                                    "type": "integer",
                                    "index": false
                                },
                                "label": {
                                    "type": "text",
                                    "index": false
                                },
                                "md_type": {
                                    "type": "integer",
                                    "index": false
                                },
                                "metadata_value_pid": {
                                    "type": "integer"
                                },
                                "pid": {
                                    "type": "integer"
                                },
                                "text_value": {
                                    "type": "keyword"
                                },
                                "unique_name": {
                                    "type": "keyword"
                                }
                            }
                        },
                        "defect": {
                            "type": "nested",
                            "properties": {
                                "aq_note": {
                                    "type": "text",
                                    "index": false
                                },
                                "associated_timestamp": {
                                    "type": "date"
                                },
                                "is_deleted": {
                                    "type": "integer"
                                },
                                "key": {
                                    "type": "keyword"
                                },
                                "last_refresh_timestamp": {
                                    "type": "date"
                                },
                                "link": {
                                    "type": "text",
                                    "index": false
                                },
                                "pid": {
                                    "type": "integer"
                                },
                                "project": {
                                    "type": "keyword"
                                },
                                "status": {
                                    "type": "keyword"
                                },
                                "test_job_reln_defect_item_pid": {
                                    "type": "integer"
                                },
                                "title": {
                                    "type": "text"
                                }
                            }
                        },
                        "end_timestamp": {
                            "type": "date"
                        },
                        "is_deleted": {
                            "type": "integer"
                        },
                        "name": {
                            "type": "keyword"
                        },
                        "pid": {
                            "type": "integer"
                        },
                        "scenario": {
                            "type": "nested",
                            "properties": {
                                "created_timestamp": {
                                    "type": "date"
                                },
                                "custom_field": {
                                    "type": "nested",
                                    "properties": {
                                        "bool_value": {
                                            "type": "integer"
                                        },
                                        "conf_entity_metadata_pid": {
                                            "type": "integer"
                                        },
                                        "date_value": {
                                            "type": "date"
                                        },
                                        "db_column_name": {
                                            "type": "keyword",
                                            "index": false
                                        },
                                        "is_required": {
                                            "type": "integer",
                                            "index": false
                                        },
                                        "label": {
                                            "type": "text",
                                            "index": false
                                        },
                                        "md_type": {
                                            "type": "integer",
                                            "index": false
                                        },
                                        "metadata_value_pid": {
                                            "type": "integer"
                                        },
                                        "pid": {
                                            "type": "integer"
                                        },
                                        "text_value": {
                                            "type": "keyword"
                                        },
                                        "unique_name": {
                                            "type": "keyword"
                                        }
                                    }
                                },
                                "is_deleted": {
                                    "type": "integer"
                                },
                                "last_modified_timestamp": {
                                    "type": "date"
                                },
                                "name": {
                                    "type": "keyword"
                                },
                                "pid": {
                                    "type": "integer"
                                },
                                "status": {
                                    "properties": {
                                        "category": {
                                            "type": "keyword"
                                        },
                                        "name": {
                                            "type": "keyword"
                                        },
                                        "pid": {
                                            "type": "integer"
                                        }
                                    }
                                },
                                "user_created": {
                                    "properties": {
                                        "is_deleted": {
                                            "type": "integer"
                                        },
                                        "name": {
                                            "type": "keyword"
                                        },
                                        "pid": {
                                            "type": "integer"
                                        }
                                    }
                                },
                                "user_last_modified": {
                                    "properties": {
                                        "is_deleted": {
                                            "type": "integer"
                                        },
                                        "name": {
                                            "type": "keyword"
                                        },
                                        "pid": {
                                            "type": "integer"
                                        }
                                    }
                                }
                            }
                        },
                        "start_timestamp": {
                            "type": "date"
                        },
                        "status": {
                            "type": "keyword"
                        }
                    }
                },
                "test_case_count": {
                    "type": "integer"
                },
                "test_info": {
                    "properties": {
                        "entity_type": {
                            "type": "keyword"
                        },
                        "name": {
                            "type": "keyword"
                        },
                        "pid": {
                            "type": "integer"
                        },
                        "test_type": {
                            "type": "keyword"
                        }
                    }
                },
                "user_created": {
                    "properties": {
                        "is_deleted": {
                            "type": "integer"
                        },
                        "name": {
                            "type": "keyword"
                        },
                        "pid": {
                            "type": "integer"
                        }
                    }
                }
            }
        }
    }
}

```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 30, 2024, 9:02am UTC](https://discuss.elastic.co/t/elastic-query-unexcepted-aggregations-doc-count/356482/2 "2024-03-30T09:02:22Z")

</div>

Wild guess: it's because you are aggregating on nested documents and the search response gives the number of the top level docs not the number of nested docs.

---

<div class="post-metadata">

**Author:** ![Vivek\_Burman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vivek_burman/32/118310_2.png) [@Vivek\_Burman](https://discuss.elastic.co/u/Vivek_Burman)\
**Post date:** [March 30, 2024, 9:08am UTC](https://discuss.elastic.co/t/elastic-query-unexcepted-aggregations-doc-count/356482/3 "2024-03-30T09:08:57Z")

</div>

I think so, since test\_case is a nested field. And each document can have multiple test\_case objects. And then further their own custom\_field and down the line. So rather than: "Give me a distribution of documents where there is a unique `test_case.custom_field.text_value` value per document"

It goes like "Give me the distribution of `test_case.custom_field.text_value` across all the documents."

---

<div class="post-metadata">

**Author:** ![Vivek\_Burman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vivek_burman/32/118310_2.png) [@Vivek\_Burman](https://discuss.elastic.co/u/Vivek_Burman)\
**Post date:** [April 1, 2024, 10:10am UTC](https://discuss.elastic.co/t/elastic-query-unexcepted-aggregations-doc-count/356482/4 "2024-04-01T10:10:45Z")

</div>

I could not explain the reason but adding a reverse nested agg at the end, resolves the issue. Any explanation would be insightful.....

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 29, 2024, 10:10am UTC](https://discuss.elastic.co/t/elastic-query-unexcepted-aggregations-doc-count/356482/5 "2024-04-29T10:10:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
