# Rollup index date\_histogram issue
**URL:** https://discuss.elastic.co/t/rollup-index-date-histogram-issue/181766
**Category:** Elasticsearch
**Created:** [May 20, 2019, 8:16am UTC](https://discuss.elastic.co/t/rollup-index-date-histogram-issue/181766 "2019-05-20T08:16:24Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@DustyMeg](https://discuss.elastic.co/u/DustyMeg)
#### Post date: [May 20, 2019, 8:16am UTC](https://discuss.elastic.co/t/rollup-index-date-histogram-issue/181766/1 "2019-05-20T08:16:25Z")
Hi,
Getting the following error when trying to Visualize a count on the Y Axis and Date Histogram of type field @timestamp with an interval of 60m on the X AxIs over Today.
`Rollup search error: [illegal_argument_exception] There is not a rollup job that has a [date_histogram] agg on field [@timestamp] which also satisfies all requirements of query.`
Running elasticsearch 7.0.0
The index pattern I am visualizing against shares both the unrolled metricbeat index and the rollup 60m interval index.
Below is the top part of the json for the rollup job.
> ```
> "config": {
> "id": "metricbeat_1h",
> "index_pattern": "metricbeat-*",
> "rollup_index": "rollup_1h_metricbeat",
> "cron": "0 15 * * * ?",
> "groups": {
> "date_histogram": {
> "interval": "60m",
> "field": "@timestamp",
> "delay": "7d",
> "time_zone": "UTC"
> },
>
> ```
I do not have the timestamp field anywhere else in the rollup json as I had believed that the date\_histogram would be enough to allow for visualizing using the timestamp field.
There is nothing showing in the elasticsearch logs.
Not sure if it is an issue with Elasticsearch or something I have set wrong.
---
### Author:  [@polyfractal](https://discuss.elastic.co/u/polyfractal)
#### Post date: [May 24, 2019, 12:27pm UTC](https://discuss.elastic.co/t/rollup-index-date-histogram-issue/181766/2 "2019-05-24T12:27:34Z")
Can you paste the (generated) query that Kibana is trying to execute?
There are basically three components that need to match when running a query:
- All the fields in the query/aggregation need to be in the job
- All the parameters are compatible; e.g. if the job was rolled up with `60m`, the aggregation needs to use a multiple of `60m`. Ditto to Timezones, etc
- All the requested aggregations were rolled up (averages, histograms, etc).
My first guess is that a parameter is not matching, like the interval or timezone.
---
### Author:  [@DustyMeg](https://discuss.elastic.co/u/DustyMeg)
#### Post date: [May 24, 2019, 12:56pm UTC](https://discuss.elastic.co/t/rollup-index-date-histogram-issue/181766/3 "2019-05-24T12:56:16Z")
Think this is the right bit.
```
{
"aggs": {
"2": {
"date_histogram": {
"field": "@timestamp",
"interval": "1h",
"time_zone": "UTC",
"min_doc_count": 1
},
"aggs": {
"1": {
"max": {
"field": "system.filesystem.used.bytes"
}
}
}
}
},
"size": 0,
"_source": {
"excludes": []
},
"stored_fields": [
"*"
],
"script_fields": {},
"docvalue_fields": [
{
"field": "@timestamp",
"format": "date_time"
}
],
"query": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"format": "strict_date_optional_time",
"gte": "2019-04-29T04:28:48.838Z",
"lte": "2019-04-29T15:58:21.157Z"
}
}
}
],
"filter": [
{
"match_all": {}
},
{
"match_all": {}
}
],
"should": [],
"must_not": []
}
}
}
```
---
### Author:  [@polyfractal](https://discuss.elastic.co/u/polyfractal)
#### Post date: [May 24, 2019, 1:05pm UTC](https://discuss.elastic.co/t/rollup-index-date-histogram-issue/181766/4 "2019-05-24T13:05:15Z")
> [@DustyMeg](#):
>
> "interval": "1h",
Ahh, I think this is the cause.
Elasticsearch has an unfortunate system for intervals at the moment. You can read more about it here (scroll down to "Calendar vs Fixed Time Intervals"): [Rollup job configuration | Elasticsearch Guide [6.7] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/6.7/rollup-job-config.html), and at the date histogram docs: [Date Histogram Aggregation | Elasticsearch Guide [6.8] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/6.8/search-aggregations-bucket-datehistogram-aggregation.html)
Basically, there are a set of units which are "calendar-aware", meaning they understand things like leap seconds, number of days or weeks per month, daylight savings time, etc. These units are singular quantities like `1h`, `1d`, `1w`, `1M`, etc.
Then there are "fixed" intervals which are strictly multiples of SI milliseconds. `2m` is `1000ms * 60 * 2`, etc.
Fixed intervals are not compatible with calendar and vice versa. So what happened is your Rollup job is defined as `60m` (fixed time), but the query is asking for `1h` (calendar time), and that's where things are going wrong. If you change the query to `60m` I think it should work.
This is definitely a very confusing situation, and something that most users don't even know about. We just [recently merged a PR](https://github.com/elastic/elasticsearch/pull/33727) to make this distinction explicit with two new fields on the date histogram agg (`calendar_interval` and `fixed_interval`), to hopefully clear this confusion up in the future.
---
### Author:  [@DustyMeg](https://discuss.elastic.co/u/DustyMeg)
#### Post date: [May 24, 2019, 1:09pm UTC](https://discuss.elastic.co/t/rollup-index-date-histogram-issue/181766/5 "2019-05-24T13:09:18Z")
Thanks will have a read.
Inside Kibana on the visualisation the interval is set to 60m, with the query outputting 1h.
If this is an issue that 7.2.0 kibana will fix. More then happy to wait till then. I have the none rollup version of this visualisation working, just not including the amount of data I was hoping for 🙂 .
---
### Author:  [@polyfractal](https://discuss.elastic.co/u/polyfractal)
#### Post date: [May 24, 2019, 1:22pm UTC](https://discuss.elastic.co/t/rollup-index-date-histogram-issue/181766/6 "2019-05-24T13:22:25Z")
> [@DustyMeg](#):
>
> nside Kibana on the visualisation the interval is set to 60m, with the query outputting 1h.
Oh, this is interesting. Might be a bug with how Kibana is generating the queries then. Lemme ping a few folks and direct them to this ticket.
The `calendar_interval`/`fixed_interval` fields were introduced in ES in 7.2, but I don't think Kibana will have support for it until 7.3 (I merged it a bit later and I don't think Kibana will have time to implement for 7.2).
Thanks for the detailed help btw! Definitely helps us identify issues easier/faster. 🙂
---
### Author:  [@DustyMeg](https://discuss.elastic.co/u/DustyMeg)
#### Post date: [June 10, 2019, 8:43am UTC](https://discuss.elastic.co/t/rollup-index-date-histogram-issue/181766/7 "2019-06-10T08:43:43Z")
Just as a quick update to this.
I was able to remidy the issue partially by adding the following in the JSON input for the X-Axis.
`{"interval": "1440m"}`
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 8, 2019, 8:43am UTC](https://discuss.elastic.co/t/rollup-index-date-histogram-issue/181766/8 "2019-07-08T08:43:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.