# Bucket Selector Aggregation on Date Histogram \_key

**URL:** <https://discuss.elastic.co/t/bucket-selector-aggregation-on-date-histogram--key/80986>\
**Category:** Elasticsearch\
**Created:** [April 3, 2017, 10:31am UTC](https://discuss.elastic.co/t/bucket-selector-aggregation-on-date-histogram--key/80986 "2017-04-03T10:31:37Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![magrossi](https://avatars.discourse-cdn.com/v4/letter/m/9d8465/32.png) [@magrossi](https://discuss.elastic.co/u/magrossi)\
**Post date:** [April 3, 2017, 10:31am UTC](https://discuss.elastic.co/t/bucket-selector-aggregation-on-date-histogram--key/80986/1 "2017-04-03T10:31:37Z")

</div>

Hi,  
I'm trying to use the bucket selector aggregation to filter out some unwanted buckets from my response, buckets that I'm not interested in (but are used in the calculation - moving avg). My main aggregation is a date\_histogram and therefore the `_key` is a date. I have tried the following without success:

```auto
    "_m_pipeline": {
      "date_histogram": {
        "field": "date",
        "interval": "month"
      },
      "aggs": {
        "_m_sum": {
          "sum": {
            "field": "value"
          }
        },
       ... <other aggregations here> ...
        "_m_data_bucket_filter": {
          "bucket_selector": {
            "buckets_path": {
              "bucket_key": "_key"
            },
            "script": "params.bucket_key < 1491004800000L" // This number is equivalent to 2017-04-01 00:00:00
          }
        }

```

The result I get is:

```auto
{
  "error": {
    "root_cause": [],
    "type": "reduce_search_phase_exception",
    "reason": "[reduce] ",
    "phase": "fetch",
    "grouped": true,
    "failed_shards": [],
    "caused_by": {
      "type": "aggregation_execution_exception",
      "reason": "buckets_path must reference either a number value or a single value numeric metric aggregation, got: org.joda.time.DateTime"
    }
  },
  "status": 503
}

```

The main idea of my query is to get a monthly moving average of window 12. In order to calculate this, I need at least 12 months of data in my histogram, but I am only interested in the latest value of the moving average, therefore I would like to discard (server side) the excess data for post processing.

Any ideas if this is possible? And how?

Thank you!

---

<div class="post-metadata">

**Author:** ![colings86](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/colings86/32/44960_2.png) [@colings86](https://discuss.elastic.co/u/colings86)\
**Post date:** [April 3, 2017, 10:52am UTC](https://discuss.elastic.co/t/bucket-selector-aggregation-on-date-histogram--key/80986/2 "2017-04-03T10:52:25Z")

</div>

Could you have a look for a stack trace in the Elasticsearch server logs and paste it here?

---

<div class="post-metadata">

**Author:** ![magrossi](https://avatars.discourse-cdn.com/v4/letter/m/9d8465/32.png) [@magrossi](https://discuss.elastic.co/u/magrossi)\
**Post date:** [April 3, 2017, 11:00am UTC](https://discuss.elastic.co/t/bucket-selector-aggregation-on-date-histogram--key/80986/3 "2017-04-03T11:00:02Z")

</div>

Hi,  
Here you go.

> [2017-04-03T10:58:32,634][DEBUG][o.e.a.s.TransportSearchAction] [fNx6XyA] failed to reduce search  
> org.elasticsearch.action.search.ReduceSearchPhaseException: [reduce]  
> at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction$2.onFailure(SearchQueryThenFetchAsyncAction.java:151) [elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.onFailure(ThreadContext.java:512) [elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:39) [elasticsearch-5.1.2.jar:5.1.2]  
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0\_121]  
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0\_121]  
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0\_121]  
> Caused by: org.elasticsearch.search.aggregations.AggregationExecutionException: buckets\_path must reference either a number value or a single value numeric metric aggregation, got: org.joda.time.DateTime  
> at org.elasticsearch.search.aggregations.pipeline.BucketHelpers.resolveBucketValue(BucketHelpers.java:171) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.pipeline.BucketHelpers.resolveBucketValue(BucketHelpers.java:152) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.pipeline.bucketselector.BucketSelectorPipelineAggregator.reduce(BucketSelectorPipelineAggregator.java:100) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.InternalAggregation.reduce(InternalAggregation.java:136) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.InternalAggregations.reduce(InternalAggregations.java:158) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.bucket.terms.InternalTerms$Bucket.reduce(InternalTerms.java:135) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.bucket.terms.InternalTerms.doReduce(InternalTerms.java:234) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.InternalAggregation.reduce(InternalAggregation.java:134) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.InternalAggregations.reduce(InternalAggregations.java:158) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.action.search.SearchPhaseController.merge(SearchPhaseController.java:489) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction$2.doRun(SearchQueryThenFetchAsyncAction.java:140) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:527) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.1.2.jar:5.1.2]  
> ... 3 more  
> [2017-04-03T10:58:32,635][WARN][r.suppressed] path: /test2/full/\_search, params: {index=test2, type=full}  
> org.elasticsearch.action.search.ReduceSearchPhaseException: [reduce]  
> at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction$2.onFailure(SearchQueryThenFetchAsyncAction.java:151) [elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.onFailure(ThreadContext.java:512) [elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:39) [elasticsearch-5.1.2.jar:5.1.2]  
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0\_121]  
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0\_121]  
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0\_121]  
> Caused by: org.elasticsearch.search.aggregations.AggregationExecutionException: buckets\_path must reference either a number value or a single value numeric metric aggregation, got: org.joda.time.DateTime  
> at org.elasticsearch.search.aggregations.pipeline.BucketHelpers.resolveBucketValue(BucketHelpers.java:171) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.pipeline.BucketHelpers.resolveBucketValue(BucketHelpers.java:152) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.pipeline.bucketselector.BucketSelectorPipelineAggregator.reduce(BucketSelectorPipelineAggregator.java:100) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.InternalAggregation.reduce(InternalAggregation.java:136) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.InternalAggregations.reduce(InternalAggregations.java:158) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.bucket.terms.InternalTerms$Bucket.reduce(InternalTerms.java:135) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.bucket.terms.InternalTerms.doReduce(InternalTerms.java:234) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.InternalAggregation.reduce(InternalAggregation.java:134) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.search.aggregations.InternalAggregations.reduce(InternalAggregations.java:158) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.action.search.SearchPhaseController.merge(SearchPhaseController.java:489) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction$2.doRun(SearchQueryThenFetchAsyncAction.java:140) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:527) ~[elasticsearch-5.1.2.jar:5.1.2]  
> at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.1.2.jar:5.1.2]  
> ... 3 more

Regards.

---

<div class="post-metadata">

**Author:** ![colings86](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/colings86/32/44960_2.png) [@colings86](https://discuss.elastic.co/u/colings86)\
**Post date:** [April 3, 2017, 2:09pm UTC](https://discuss.elastic.co/t/bucket-selector-aggregation-on-date-histogram--key/80986/4 "2017-04-03T14:09:34Z")

</div>

This seems to be a bug unfortunately. I have opened [https://github.com/elastic/elasticsearch/issues/23874](https://github.com/elastic/elasticsearch/issues/23874) to get the problem fixed but basically we have a requirement at the moment that all bucket paths resolve to a numeric value (as in an instance of `java.lang.Number`). This is important for most of the pipeline aggregations since, by design, they work only on numeric values. However in the case of `bucket_script` and `bucket_selector` we should allow other types so long as the script can support them (essentially this means allowing `DateTime` and `String` values as well).

Unfortunately I can't think of a way to work around this at the moment, sorry.

---

<div class="post-metadata">

**Author:** ![magrossi](https://avatars.discourse-cdn.com/v4/letter/m/9d8465/32.png) [@magrossi](https://discuss.elastic.co/u/magrossi)\
**Post date:** [April 3, 2017, 2:24pm UTC](https://discuss.elastic.co/t/bucket-selector-aggregation-on-date-histogram--key/80986/5 "2017-04-03T14:24:02Z")

</div>

Thanks for the quick feedback. I'll follow that opened issue closely but will have to deal with this on client side for now I guess.  
Regards.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 1, 2017, 2:24pm UTC](https://discuss.elastic.co/t/bucket-selector-aggregation-on-date-histogram--key/80986/6 "2017-05-01T14:24:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.