When are “buckets”: [] in an aggregation?

[WoJ]

My query is a nested aggregation

aggs: {
    src: {
        terms: {
            field: "dst_ip",
            size: 1000,
        },
        aggs: {
            dst: {
                terms: {
                    field: "a_field_which_changes",
                    size: 2000,
                },
            },
        },
    },

A typical doc the query is ran against is below (the mappings are all of type keyword)

{
    "_index": "honey",
    "_type": "event",
    "_id": "AWHzRjHrjNgIX_EoDcfV",
    "_score": 1,
    "_source": {
      "dst_ip": "10.101.146.166",
      "src_ip": "10.10.16.1",
      "src_port": "38",
    }
},

There are actually two queries I make, one after the other. They differ by the value of a_field_which_changes, which is "src_ip" in one query and "src_port" in the other.

In the first query all the results are fine. The aggregation is 1 element large and the buckets specify what that element matched with

{
    "key": "10.6.17.218",      <--- "dst_ip" field
    "doc_count": 1,
    "dst": {
    "doc_count_error_upper_bound": 0,
    "sum_other_doc_count": 0,
    "buckets": [
        {
        "key": "-1",       <--- "src_port" field
        "doc_count": 1
        }
    ]
    }
},

The other query yields two different kind of results:

{
    "key": "10.6.17.218",
    "doc_count": 1,
    "dst": {
    "doc_count_error_upper_bound": 0,
    "sum_other_doc_count": 0,
    "buckets": []
    }
},
{
      "key": "10.237.78.19",
      "doc_count": 1,
      "dst": {
        "doc_count_error_upper_bound": 0,
        "sum_other_doc_count": 0,
        "buckets": [
          {
            "key": "10.12.67.89",
            "doc_count": 1
          }
        ]
      }
 },

The first result is problematic: it does not give the details of the buckets. It is no different from the other one but somehow the details are missing.

Why is it so, and most importantly - how to force Elasticsearch to display the details of the buckets?

The documentation goes into details on how to interfere with the aggregation but I could not find anything relevant there.

(initially asked on SO)