Aggregations on date fields

[stevenmwall (Steve Wall)]

I am trying to create an aggregation on a date field (in my case, named "startDate") that will put documents into buckets based upon the startDate month (while ignoring the year). For example, documents with a start date of 2017-Jan-30 and 2016-Jan-12 should go into the same bucket.

Ultimately I would like my aggregation response to be something like this:

  "aggregations": {
    "startDateByMonth": {
      "buckets": [
        {
          "key_as_string": "01",
          "key": "January",
          "doc_count": 3
        },
        {
          "key_as_string": "02",
          "key": "February",
          "doc_count": 1
        },
        {
          "key_as_string": "03",
          "key": "March",
          "doc_count": 6
        },
        //Other months with doc_counts
}

I have tried using the Date Histogram aggregation but I don't think it is quite what I need, or at least I'm not seeing what I need to do.

I would appreciate if anyone has any ideas.

Thanks,
Steve

[spinscale (Alexander Reelsen)]

Hey,

I see two solutions here. One one query time and one on index time.

Query time

Use a script in your aggregation to extract the month of a date. This however means to run for each document

GET logs/_search
{
  "size": 0,
  "aggs": {
    "per-month": {
      "terms": {
        "script":  { "inline" : "return doc['date'].getDate().getMonthOfYear()" }
      }
    }
  }
}

Index time

how about using an ingest processor to write the month on index time and then simply run a terms aggregations on query (which is much faster). Check out this pipeline

POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [
      {
        "script": {
          "inline": "ctx.month = LocalDateTime.parse(ctx.date).getMonth()"
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "date": "2017-02-26T00:00"
      }
    }
  ]
}

Creating the above pipeline and adding it to your index operations will always create your field of the month in each document.

--Alex