# Sum\_other\_doc\_count higher than total docs

**URL:** <https://discuss.elastic.co/t/sum-other-doc-count-higher-than-total-docs/140490>\
**Category:** Elasticsearch\
**Created:** [July 18, 2018, 9:15am UTC](https://discuss.elastic.co/t/sum-other-doc-count-higher-than-total-docs/140490 "2018-07-18T09:15:10Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![covolution](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/covolution/32/22840_2.png) [@covolution](https://discuss.elastic.co/u/covolution)\
**Post date:** [July 18, 2018, 9:15am UTC](https://discuss.elastic.co/t/sum-other-doc-count-higher-than-total-docs/140490/1 "2018-07-18T09:15:11Z")

</div>

I have a single shard index with 1,464,467 docs. I am doing some aggregations on a keyword field. The terms aggregation is returning a "sum\_other\_doc\_count": 1637376, which is more documents than I have in the index. I am trying to understand how it can be so inaccurate?

```GET
{
    "aggs" : {
        "cardinality_count" : { "cardinality" : { "field" : "dc:subjects" } },
        "missing_count" : { "missing" : { "field" : "dc:subjects" } }
    },
    "size" : 0
}

```

Returns:

```auto
        "took": 597,
        "timed_out": false,
        "_shards": {
            "total": 1,
            "successful": 1,
            "skipped": 0,
            "failed": 0
        },
        "hits": {
            "total": 1464467,
            "max_score": 0,
            "hits": []
        },
        "aggregations": {
            "cardinality_count": {
                "value": 105103
            },
            "missing_count": {
                "doc_count": 501609
            }
        }
    }

```

If I want to get the top 50 terms from my dataset my query is:

```auto
      "size": 0,
      "aggs": {
        "my_terms": {
          "terms": {
            "field": "dc:subjects",
            "size": 50
          }
        }
      }
    }

```

And the partial result is:

```auto
    "took": 128,
    "timed_out": false,
    "hits": {
        "total": 1464467,
    },
    "aggregations": {
        "my_terms": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 1637376,
            "buckets": [
                {
                    "key": "World War, 1939-1945",
                    "doc_count": 16259
                },
                {
                    "key": "Women",
                    "doc_count": 10369
                },
                {
                    "key": "Education",
                    "doc_count": 6699
                },
                {
                    "key": "Law",
                    "doc_count": 5248
                },
                {
                    "key": "Architecture",
                    "doc_count": 5018
                },
                {
                    "key": "Music",
                    "doc_count": 4813
                },

```

I have read the documentation saying the counts can be inaccurate. I would like to understand how "sum\_other\_doc\_count" can be 1637376 (for 1 shard). I realise the cardinality is high but I don't think its excessive for more than a million docs.

> "sum\_other\_doc\_count": this number is the sum of the document counts for all buckets that are not part of the response.

What am I not understanding?

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [July 18, 2018, 7:20pm UTC](https://discuss.elastic.co/t/sum-other-doc-count-higher-than-total-docs/140490/2 "2018-07-18T19:20:24Z")

</div>

Do your documents have more than one value for `dc:subjects`? If so, a single document can be in multiple buckets for the terms aggregation. In other words, a document will be counted multiple times, if it has multiple values for `dc:subjects` (once for each value it has for `dc:subjects`).

That's how you end up with a `sum_other_doc_count` that is higher than your number of documents - the sum of the counts of all the remaining buckets (outside of the top 50 buckets) adds up to more than the number of documents.

---

<div class="post-metadata">

**Author:** ![covolution](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/covolution/32/22840_2.png) [@covolution](https://discuss.elastic.co/u/covolution)\
**Post date:** [July 19, 2018, 9:16am UTC](https://discuss.elastic.co/t/sum-other-doc-count-higher-than-total-docs/140490/3 "2018-07-19T09:16:47Z")

</div>

I just checked my data. You are right, its a multi-valued field. Make perfect sense, thanks for your help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 16, 2018, 9:16am UTC](https://discuss.elastic.co/t/sum-other-doc-count-higher-than-total-docs/140490/4 "2018-08-16T09:16:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.