Inconsistent results in aggregations (Or ghost records)

Elastic Stack Elasticsearch
1

Hi running 2.1.0

I'm executing this aggregation

GET my index/_search
{
  "size": 0,
  "aggs": {
    "by_date": {
      "date_histogram": {
        "field": "my_date",
        "interval": "day"
      }
    }
  }
}

The search returns ghost records. If I recall this is because the replica shards are not consistent with the primary shards?

The index is 32 shards with 1 replica. So 64 shards total.

This is supposed to be the right result.

  "buckets": [
    {
      "key_as_string": "2016-02-05T00:00:00.000Z",
      "key": 1454630400000,
      "doc_count": 2761324
    }
  ]

This is the ghost results.

"buckets": [
        {
          "key_as_string": "2016-02-03T00:00:00.000Z",
          "key": 1454457600000,
          "doc_count": 6494
        },
        {
          "key_as_string": "2016-02-04T00:00:00.000Z",
          "key": 1454544000000,
          "doc_count": 0
        },
        {
          "key_as_string": "2016-02-05T00:00:00.000Z",
          "key": 1454630400000,
          "doc_count": 2751324
        }
      ]