Custom date histogram interval

Elastic Stack Elasticsearch
1

Is there any way to use a custom interval with date histograms (either
facets or aggregations)? For example, something like

{
  "date_histogram": {
    "field": "date",
    "interval": "fiscalYear"
  }
}

Obviously, you can always use a regular histogram, but then the client
would have to manage all the complexities that come with dealing with time.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/37ef7076-f68f-49ff-bd48-3d8431a576d1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Hello Gabe ,

Please elaborate on what you mean by custom interval .

Thanks
Vineeth

On Tue, Jul 8, 2014 at 11:25 PM, Gabe Gorelick-Feldman <
gabegorelick@gmail.com> wrote:

Is there any way to use a custom interval with date histograms (either
facets or aggregations)? For example, something like

{
  "date_histogram": {
    "field": "date",
    "interval": "fiscalYear"
  }
}

Obviously, you can always use a regular histogram, but then the client
would have to manage all the complexities that come with dealing with time.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/37ef7076-f68f-49ff-bd48-3d8431a576d1%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/37ef7076-f68f-49ff-bd48-3d8431a576d1%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGdPd5kN788jMHficcQbHa%2BtEsFkn34h%3DUVhqEeRG_NJx_gguA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Sure. According to the docs [1], the available expressions for
date_histogram interval are year, quarter, month, week, day, hour, minute,
second. But what if you want to rollup by another interval that's not
supported, like decade or millisecond? I was just wondering if there was
maybe a way to write a plugin for such custom intervals.

Specifically, I need a CDC MMWR week [2] interval, which is similar to ISO
week (which I assume is what elasticsearch uses), but not exactly the same.

[1] Elasticsearch Platform — Find real-time answers at scale | Elastic
[2] http://wwwn.cdc.gov/nndss/document/MMWR_Week_overview.pdf

On Tuesday, July 8, 2014 5:09:39 PM UTC-4, vineeth mohan wrote:

Hello Gabe ,

Please elaborate on what you mean by custom interval .

Thanks
Vineeth

On Tue, Jul 8, 2014 at 11:25 PM, Gabe Gorelick-Feldman <
gabego...@gmail.com <javascript:>> wrote:

Is there any way to use a custom interval with date histograms (either
facets or aggregations)? For example, something like

{
  "date_histogram": {
    "field": "date",
    "interval": "fiscalYear"
  }
}

Obviously, you can always use a regular histogram, but then the client
would have to manage all the complexities that come with dealing with time.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/37ef7076-f68f-49ff-bd48-3d8431a576d1%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/37ef7076-f68f-49ff-bd48-3d8431a576d1%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b8617abb-ed9e-4fff-95f6-cb81a3def727%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Hello Gabe ,

You can mention the range as 1.5h or 2w and all -

Hope that might help your cause.

Thanks
Vineeth

On Wed, Jul 9, 2014 at 2:49 AM, Gabe Gorelick-Feldman <
gabegorelick@gmail.com> wrote:

Sure. According to the docs [1], the available expressions for
date_histogram interval are year, quarter, month, week, day, hour, minute,
second. But what if you want to rollup by another interval that's not
supported, like decade or millisecond? I was just wondering if there was
maybe a way to write a plugin for such custom intervals.

Specifically, I need a CDC MMWR week [2] interval, which is similar to ISO
week (which I assume is what elasticsearch uses), but not exactly the same.

[1]
Elasticsearch Platform — Find real-time answers at scale | Elastic
[2] http://wwwn.cdc.gov/nndss/document/MMWR_Week_overview.pdf

On Tuesday, July 8, 2014 5:09:39 PM UTC-4, vineeth mohan wrote:

Hello Gabe ,

Please elaborate on what you mean by custom interval .

Thanks
Vineeth

On Tue, Jul 8, 2014 at 11:25 PM, Gabe Gorelick-Feldman <
gabego...@gmail.com> wrote:

Is there any way to use a custom interval with date histograms (either
facets or aggregations)? For example, something like

{
  "date_histogram": {
    "field": "date",
    "interval": "fiscalYear"
  }
}

Obviously, you can always use a regular histogram, but then the client
would have to manage all the complexities that come with dealing with time.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/37ef7076-f68f-49ff-bd48-3d8431a576d1%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/37ef7076-f68f-49ff-bd48-3d8431a576d1%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b8617abb-ed9e-4fff-95f6-cb81a3def727%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b8617abb-ed9e-4fff-95f6-cb81a3def727%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGdPd5mNw06nSfLdjDV%2BYK829AR%2Bxk2sKZmerVvjyEbg7HWPjA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

5

That solves the decade or millisecond problem, but wouldn't work for
something like "MMWR week" or "fiscal year" which are more complex. Here's
the CDC's definition of MMWR week to illustrate my point:

The first day of any MMWR week is Sunday. MMWR week numbering is sequential

beginning with 1 and
incrementing with each week to a maximum of 52 or 53. MMWR week #1 of an
MMWR year is the first week of
the year that has at least four days in the calendar year. For example, if
January 1 occurs on a Sunday, Monday,
Tuesday or Wednesday, the calendar week that includes January 1 would be
MMWR week #1. If January 1
occurs on a Thursday, Friday, or Saturday, the calendar week that includes
January 1 would be the last MMWR
week of the previous year (#52 or #53). Because of this rule, December 29,
30, and 31 could potentially fall into
MMWR week #1 of the following MMWR year.

On Tuesday, July 8, 2014 5:29:54 PM UTC-4, vineeth mohan wrote:

Hello Gabe ,

You can mention the range as 1.5h or 2w and all -
Elasticsearch Platform — Find real-time answers at scale | Elastic

Hope that might help your cause.

Thanks
Vineeth

On Wed, Jul 9, 2014 at 2:49 AM, Gabe Gorelick-Feldman <gabego...@gmail.com
<javascript:>> wrote:

Sure. According to the docs [1], the available expressions for
date_histogram interval are year, quarter, month, week, day, hour, minute,
second. But what if you want to rollup by another interval that's not
supported, like decade or millisecond? I was just wondering if there was
maybe a way to write a plugin for such custom intervals.

Specifically, I need a CDC MMWR week [2] interval, which is similar to
ISO week (which I assume is what elasticsearch uses), but not exactly the
same.

[1]
Elasticsearch Platform — Find real-time answers at scale | Elastic
[2] http://wwwn.cdc.gov/nndss/document/MMWR_Week_overview.pdf

On Tuesday, July 8, 2014 5:09:39 PM UTC-4, vineeth mohan wrote:

Hello Gabe ,

Please elaborate on what you mean by custom interval .

Thanks
Vineeth

On Tue, Jul 8, 2014 at 11:25 PM, Gabe Gorelick-Feldman <
gabego...@gmail.com> wrote:

Is there any way to use a custom interval with date histograms
(either facets or aggregations)? For example, something like

{
  "date_histogram": {
    "field": "date",
    "interval": "fiscalYear"
  }
}

Obviously, you can always use a regular histogram, but then the client
would have to manage all the complexities that come with dealing with time.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/37ef7076-f68f-49ff-bd48-3d8431a576d1%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/37ef7076-f68f-49ff-bd48-3d8431a576d1%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b8617abb-ed9e-4fff-95f6-cb81a3def727%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b8617abb-ed9e-4fff-95f6-cb81a3def727%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9c905198-0926-4cd9-ac43-438c29fa59a6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

6

Hello Gabe ,

Th only thing , I can think of would be to store the fiscal year date as a
separate field while indexing.
And then do all manipulation on this date.

Thanks
Vineeth

On Wed, Jul 9, 2014 at 3:10 AM, Gabe Gorelick-Feldman <
gabegorelick@gmail.com> wrote:

That solves the decade or millisecond problem, but wouldn't work for
something like "MMWR week" or "fiscal year" which are more complex. Here's
the CDC's definition of MMWR week to illustrate my point:

The first day of any MMWR week is Sunday. MMWR week numbering is

sequential beginning with 1 and
incrementing with each week to a maximum of 52 or 53. MMWR week #1 of an
MMWR year is the first week of
the year that has at least four days in the calendar year. For example,
if January 1 occurs on a Sunday, Monday,
Tuesday or Wednesday, the calendar week that includes January 1 would be
MMWR week #1. If January 1
occurs on a Thursday, Friday, or Saturday, the calendar week that
includes January 1 would be the last MMWR
week of the previous year (#52 or #53). Because of this rule, December
29, 30, and 31 could potentially fall into
MMWR week #1 of the following MMWR year.

On Tuesday, July 8, 2014 5:29:54 PM UTC-4, vineeth mohan wrote:

Hello Gabe ,

You can mention the range as 1.5h or 2w and all -
Elasticsearch Platform — Find real-time answers at scale | Elastic
reference/current/search-aggregations-bucket-
datehistogram-aggregation.html

Hope that might help your cause.

Thanks
Vineeth

On Wed, Jul 9, 2014 at 2:49 AM, Gabe Gorelick-Feldman <
gabego...@gmail.com> wrote:

Sure. According to the docs [1], the available expressions for
date_histogram interval are year, quarter, month, week, day, hour, minute,
second. But what if you want to rollup by another interval that's not
supported, like decade or millisecond? I was just wondering if there was
maybe a way to write a plugin for such custom intervals.

Specifically, I need a CDC MMWR week [2] interval, which is similar to
ISO week (which I assume is what elasticsearch uses), but not exactly the
same.

[1] Elasticsearch Platform — Find real-time answers at scale | Elastic
reference/current/search-aggregations-bucket-
datehistogram-aggregation.html
[2] http://wwwn.cdc.gov/nndss/document/MMWR_Week_overview.pdf

On Tuesday, July 8, 2014 5:09:39 PM UTC-4, vineeth mohan wrote:

Hello Gabe ,

Please elaborate on what you mean by custom interval .

Thanks
Vineeth

On Tue, Jul 8, 2014 at 11:25 PM, Gabe Gorelick-Feldman <
gabego...@gmail.com> wrote:

Is there any way to use a custom interval with date histograms
(either facets or aggregations)? For example, something like

{
  "date_histogram": {
    "field": "date",
    "interval": "fiscalYear"
  }
}

Obviously, you can always use a regular histogram, but then the client
would have to manage all the complexities that come with dealing with time.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/37ef7076-f68f-49ff-bd48-3d8431a576d1%40goo
glegroups.com
https://groups.google.com/d/msgid/elasticsearch/37ef7076-f68f-49ff-bd48-3d8431a576d1%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/b8617abb-ed9e-4fff-95f6-cb81a3def727%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b8617abb-ed9e-4fff-95f6-cb81a3def727%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9c905198-0926-4cd9-ac43-438c29fa59a6%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9c905198-0926-4cd9-ac43-438c29fa59a6%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGdPd5n8JeYxGZAQ_SjMdHaUycGm2nSGwJeqKGJGUp%2BTZg2gzA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

7

Thanks, I might do that. Just wanted to make sure there wasn't some easy
way to have an "interval script"

On Tuesday, July 8, 2014 5:53:39 PM UTC-4, vineeth mohan wrote:

Hello Gabe ,

Th only thing , I can think of would be to store the fiscal year date as a
separate field while indexing.
And then do all manipulation on this date.

Thanks
Vineeth

On Wed, Jul 9, 2014 at 3:10 AM, Gabe Gorelick-Feldman <gabego...@gmail.com
<javascript:>> wrote:

That solves the decade or millisecond problem, but wouldn't work for
something like "MMWR week" or "fiscal year" which are more complex. Here's
the CDC's definition of MMWR week to illustrate my point:

The first day of any MMWR week is Sunday. MMWR week numbering is

sequential beginning with 1 and
incrementing with each week to a maximum of 52 or 53. MMWR week #1 of an
MMWR year is the first week of
the year that has at least four days in the calendar year. For example,
if January 1 occurs on a Sunday, Monday,
Tuesday or Wednesday, the calendar week that includes January 1 would be
MMWR week #1. If January 1
occurs on a Thursday, Friday, or Saturday, the calendar week that
includes January 1 would be the last MMWR
week of the previous year (#52 or #53). Because of this rule, December
29, 30, and 31 could potentially fall into
MMWR week #1 of the following MMWR year.

On Tuesday, July 8, 2014 5:29:54 PM UTC-4, vineeth mohan wrote:

Hello Gabe ,

You can mention the range as 1.5h or 2w and all -
Elasticsearch Platform — Find real-time answers at scale | Elastic
reference/current/search-aggregations-bucket-
datehistogram-aggregation.html

Hope that might help your cause.

Thanks
Vineeth

On Wed, Jul 9, 2014 at 2:49 AM, Gabe Gorelick-Feldman <
gabego...@gmail.com> wrote:

Sure. According to the docs [1], the available expressions for
date_histogram interval are year, quarter, month, week, day, hour, minute,
second. But what if you want to rollup by another interval that's not
supported, like decade or millisecond? I was just wondering if there was
maybe a way to write a plugin for such custom intervals.

Specifically, I need a CDC MMWR week [2] interval, which is similar to
ISO week (which I assume is what elasticsearch uses), but not exactly the
same.

[1] Elasticsearch Platform — Find real-time answers at scale | Elastic
reference/current/search-aggregations-bucket-
datehistogram-aggregation.html
[2] http://wwwn.cdc.gov/nndss/document/MMWR_Week_overview.pdf

On Tuesday, July 8, 2014 5:09:39 PM UTC-4, vineeth mohan wrote:

Hello Gabe ,

Please elaborate on what you mean by custom interval .

Thanks
Vineeth

On Tue, Jul 8, 2014 at 11:25 PM, Gabe Gorelick-Feldman <
gabego...@gmail.com> wrote:

Is there any way to use a custom interval with date histograms
(either facets or aggregations)? For example, something like

{
  "date_histogram": {
    "field": "date",
    "interval": "fiscalYear"
  }
}

Obviously, you can always use a regular histogram, but then the
client would have to manage all the complexities that come with dealing
with time.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/37ef7076-f68f-49ff-bd48-3d8431a576d1%40goo
glegroups.com
https://groups.google.com/d/msgid/elasticsearch/37ef7076-f68f-49ff-bd48-3d8431a576d1%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/b8617abb-ed9e-4fff-95f6-cb81a3def727%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b8617abb-ed9e-4fff-95f6-cb81a3def727%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9c905198-0926-4cd9-ac43-438c29fa59a6%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9c905198-0926-4cd9-ac43-438c29fa59a6%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e99adff6-d7d1-40db-a4db-998ba10347ae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.