Date_histogram and day buckets

Elastic Stack Elasticsearch
1

Hi,

Previously on v5 when using the following query we would only be returned the buckets that are within the range specified. We need the min buckets to be able to graph correctly.

On v7.3 the same query returns a bucket for every day since 1/1/1970. I've struggled to work out how to disregard the buckets outside the range, or even why it is now doing this.

Any help/advice would be much appreciated as the docs aren't offering up any answers.

{
   "query" : {
      "bool" : {
         "filter" : [
            {
               "range" : {
                  "StartField" : {
                     "gte" : 1567292400,
                     "lte" : 1569884399,
                     "format" : "epoch_second"
                  }
               }
            }
         ]
      }
   },
   "size" : 0,
   "aggs" : {
      "accountTotals" : {
         "terms" : {
            "size" : 10,
            "field" : "account"
         },
         "aggs" : {
            "periods" : {
               "date_histogram" : {
                  "field" : "StartField",
                  "interval" : "day",
                  "format" : "dd-MM-YYYY",
                  "min_doc_count": 0,
                  "extended_bounds" : {
                     "min" : "01-09-2019",
                     "max" : "30-09-2019"
                  }
               },
               "aggs" : {
                  "Totals" : {
                     "value_count" : {
                        "field" : "messageId"
                     }
                  }
               }
            }
         }
      }
   }
}
2

Just thought I'd also post a simpler example that shows the same issue, results all the way from 1970, outside of the bounds requested

{
   "size" : 0,
    "aggs" : {
        "by_day" : {
            "date_histogram" : {
                "field" : "StartField",
                "format" : "dd-MM-YYYY",
                "interval" : "day",
                  "extended_bounds" : {
                     "min" : "01-09-2019",
                     "max" : "30-09-2019"
                  }
            }
        }
    }
}
3

You need to provide a format with lower case ys instead of upper case Ys.

Like this:

{
  "size": 0,
  "aggs": {
    "by_day": {
      "date_histogram": {
        "field": "@timestamp",
        "format": "dd-MM-yyyy",
        "interval": "day",
        "extended_bounds": {
          "min": "01-09-2019",
          "max": "30-09-2019"
        }
      }
    }
  }
}

You can find the syntax for custom date formats in the documentation for DateTimeFormatter .

1 Like
4

So simple and yet so easy to miss. Thank you kindly!

I need to dig deeper though as this worked correctly in V5

5

Elasticsearch switched date libraries from v6 to v7. Some date formats have changed as a result of that. This seems to be one of those changes, that caused your format to work in v5, but not in v7.

6

Thank you for the info

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.