# ES 6.1.0 Aggregations Empty

**URL:** <https://discuss.elastic.co/t/es-6-1-0-aggregations-empty/112100>\
**Category:** Elasticsearch\
**Created:** [December 16, 2017, 7:02pm UTC](https://discuss.elastic.co/t/es-6-1-0-aggregations-empty/112100 "2017-12-16T19:02:36Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![shreyask](https://avatars.discourse-cdn.com/v4/letter/s/ecc23a/32.png) [@shreyask](https://discuss.elastic.co/u/shreyask)\
**Post date:** [December 16, 2017, 7:02pm UTC](https://discuss.elastic.co/t/es-6-1-0-aggregations-empty/112100/1 "2017-12-16T19:02:36Z")

</div>

I have about 400K documents in ES (3 node cluster), the documents have following format:

```
{
    "@timestamp": "2017-11-21T06:49:39.800Z",
    "message": "xxxx",
    "username": "xxxx",
    "geoip": {
        "country_name": "United States",
        "region_name": "Washington",
        "city_name": "Seattle"
    }
}

```

I am performing nested aggregations using the following query:

```
{
    "size": 0,
    "query": {
        "bool": {
            "must": [
                {
                    "range": {
                        "@timestamp": {
                            "gte": "now-90d/d",
                            "lt": "now/d",
                            "time_zone": "US/Pacific"
                        }
                    }
                }
            ]
        }
    },
    "sort": [
        {
            "@timestamp": {
                "order": "desc"
            }
        }
    ],
    "aggs": {
        "countries": {
            "terms": {
                "size": 1000000,
                "field": "geoip.country_name.keyword"
            },
            "aggs": {
                "regions": {
                    "terms": {
                        "size": 1000000,
                        "field": "geoip.region_name.keyword"
                    },
                    "aggs": {
                        "cities": {
                            "terms": {
                                "size": 1000000,
                                "field": "geoip.city_name.keyword.keyword"
                            },
                            "aggs": {
                                "users": {
                                    "terms": {
                                        "size": 1000000,
                                        "field": "username.keyword"
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

```

I do get results back for this query and the aggregations are computed. However if I change the time range to target some of the most recent documents for example:

```
{
    "range": {
        "@timestamp": {
            "gte": "now-2d/d",
            "lt": "now/d",
            "time_zone": "US/Pacific"
        }
    }
}

```

I get `hits` back but no aggregations, I can confirm that there is documents matching the time range I provided and the data format has not changed either.

Only thing I can think of is that I recently upgraded to `ES 6.1.0` from `ES 6.0.0`  
I found this odd that ES computes aggregations for slice of documents containing older documents but does not compute aggregations for newer documents.

Can anyone point me in the right direction to solve this issue?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 16, 2017, 7:17pm UTC](https://discuss.elastic.co/t/es-6-1-0-aggregations-empty/112100/2 "2017-12-16T19:17:16Z")

</div>

What if you remove

```
"size": 100000

```

For each agg?

---

<div class="post-metadata">

**Author:** ![shreyask](https://avatars.discourse-cdn.com/v4/letter/s/ecc23a/32.png) [@shreyask](https://discuss.elastic.co/u/shreyask)\
**Post date:** [December 16, 2017, 8:09pm UTC](https://discuss.elastic.co/t/es-6-1-0-aggregations-empty/112100/3 "2017-12-16T20:09:59Z")

</div>

I just tried that, still get empty aggregations:

```
{
  "took": 196,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 20266844,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "countries": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": []
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 19, 2017, 9:10am UTC](https://discuss.elastic.co/t/es-6-1-0-aggregations-empty/112100/4 "2017-12-19T09:10:42Z")

</div>

Are you running the query on multiple indices? like `index-*`?

I wonder if the mapping is different for most recent indices?

Could you check the mapping you have for a recent index?  
And if it looks ok to you, could you reproduce a small script like:

```auto
DELETE index
PUT index
{
  "settings": {},
  "mappings": {}
}
PUT index/doc/1
{
    "@timestamp": "2017-11-21T06:49:39.800Z",
    "message": "xxxx",
    "username": "xxxx",
    "geoip": {
        "country_name": "United States",
        "region_name": "Washington",
        "city_name": "Seattle"
    }
}
GET index/_search
{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "@timestamp": {
              "gte": "now-2d/d",
              "lt": "now/d",
              "time_zone": "US/Pacific"
            }
          }
        }
      ]
    }
  },
  "sort": [
    {
      "@timestamp": {
        "order": "desc"
      }
    }
  ],
  "aggs": {
    "countries": {
      "terms": {
        "size": 1000000,
        "field": "geoip.country_name.keyword"
      },
      "aggs": {
        "regions": {
          "terms": {
            "size": 1000000,
            "field": "geoip.region_name.keyword"
          },
          "aggs": {
            "cities": {
              "terms": {
                "size": 1000000,
                "field": "geoip.city_name.keyword.keyword"
              },
              "aggs": {
                "users": {
                  "terms": {
                    "size": 1000000,
                    "field": "username.keyword"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![shreyask](https://avatars.discourse-cdn.com/v4/letter/s/ecc23a/32.png) [@shreyask](https://discuss.elastic.co/u/shreyask)\
**Post date:** [December 19, 2017, 6:54pm UTC](https://discuss.elastic.co/t/es-6-1-0-aggregations-empty/112100/5 "2017-12-19T18:54:09Z")

</div>

> [@dadoonet](#):
>
> Are you running the query on multiple indices? like index-\*?

No, I am not all of the data is in one single index.

Here is the mapping from the index (from `/app-access/_mappings`)

```
{
    "app-access": {
        "mappings": {
            "doc": { 
              // ES 6.0 onwards
                "_all": {
                    "enabled": true
                },
                "properties": {
                    "@timestamp": {
                        "type": "date"
                    },
                    "geoip": {
                        "properties": {
                            "city_name": {
                                "type": "text"
                            },
                            "country_name": {
                                "type": "text"
                            },
                            "region_name": {
                                "type": "text"
                            }
                        }
                    }
                },
                "app-access": {
                   // mapping ES 5.6.X
                    "_all": {
                        "enabled": true
                    },
                    "properties": {
                        "@timestamp": {
                            "type": "date"
                        },
                        "geoip": {
                            "properties": {
                                "city_name": {
                                    "type": "text",
                                    "fields": {
                                        "keyword": {
                                            "type": "keyword",
                                            "ignore_above": 256
                                        }
                                    }
                                },
                                "country_name": {
                                    "type": "text",
                                    "fields": {
                                        "keyword": {
                                            "type": "keyword",
                                            "ignore_above": 256
                                        }
                                    }
                                },
                                "region_name": {
                                    "type": "text",
                                    "fields": {
                                        "keyword": {
                                            "type": "keyword",
                                            "ignore_above": 256
                                        }
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

```

Upon close inspection I see that when the upgrade to ES 6.0 took place a new mapping under the type doc was created and the earlier mapping which had the `"type: keyword"` and now new documents (post ES 6.X) do not have the `"type: keyword"` hence the aggregations are empty for the latest documents (I guess)

So to provide more context I am using the `logstash-elasticsearch-output` plugin to ingest the logs and since ES 6.X the multiple `_types` not supported I removed explicit `document_type => "access_logs"` from my logstash config and I can see why this mapping issue happened.

So at this point shall I use the `re-index` API to fix this issue or is there is prescribed way to solve this?

I am thankful for the great pointer regarding checking out mapping for this index.

---

<div class="post-metadata">

**Author:** ![shreyask](https://avatars.discourse-cdn.com/v4/letter/s/ecc23a/32.png) [@shreyask](https://discuss.elastic.co/u/shreyask)\
**Post date:** [December 19, 2017, 7:50pm UTC](https://discuss.elastic.co/t/es-6-1-0-aggregations-empty/112100/6 "2017-12-19T19:50:54Z")

</div>

Thanks @dadoonet for pointing towards `mappings` I was able to use the `reindex` API to fix this issue.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 16, 2018, 7:50pm UTC](https://discuss.elastic.co/t/es-6-1-0-aggregations-empty/112100/7 "2018-01-16T19:50:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
