When trying to do calculations with two previously fetched/aggregated (using date_range) fields I'm stumbling upon a Null Pointer Exception error whenever I use bucket_script. The strange thing is that the request below works perfectly fine when replacing date_range with date_histogram or a terms aggregation. The moment I fill in anything in the script field in my bucket_script I get a NPE as response.
The error message I'm getting as response:
"{"error":{"root_cause":,"type":"reduce_search_phase_exception","reason":"[reduce] ","phase":"fetch","grouped":true,"failed_shards":,"caused_by":{"type":"null_pointer_exception","reason":null}},"status":503}"
My JSON request:
{ "query": { "bool": "must": [ { "query_string": { "default_field": "data.adgroup_id", "query": "(adgroup_id: 6044965376773 OR adgroup_id: 6045143401573 OR adgroup_id: 6044653493573 OR adgroup_id: 6044738299773)" } }, { "term": { "agency_id": "215" } }, { "range" : { "date" : { "gte" : "2016-06-01", "lte" : "2016-06-30" } } } ], "must_not": [], "should": [] } }, "from": 0, "size": 50, "sort": [], "aggs": { "indicators": { "date_range": { "field": "date", "format": "yyyy-MM-dd", "ranges": [ { "from": "2016-06-01" }, { "to": "2016-06-30" } ] }, "aggs": { "ind1_c_website_clicks": { "sum": { "field": "data.website_clicks"} },"ind1_c_impressions": { "sum": { "field": "data.impressions "} } ,"ind1_ctr" : { "bucket_script": { "gap_policy" : "insert_zeros", "buckets_path": { "var_website_clicks" : "ind1_c_website_clicks", "var_impressions" : "ind1_c_impressions" }, "script": "var_website_clicks / var_impressions * 100" } } } } } }
If I leave out the bucket_script segment I get valid results for the fields ind1_c_website_clicks and ind1_c_impressions and I just want to do some direct calculations with those results. When changing the field names in buckets_path to gibberish I get some other error regarding ES being unable to find that field, which to me seems like it doesn't have anything to do with the path reference being incorrect. Putting 0 or random() in script to return a direct result also causes the NPE, so it doesn't seem to have anything to do with fields not being found or values being 0 / null.
I'm guessing it has something to do with the structure of the aggregations and the way date_range accesses data. The pipeline aggregation doc provided some insights about parent and sibling buckets and I've tried moving ind1_ctr out of the aggregations {} since it seemed weird (I've copied this structuring according to one of the bucket script example) to me that this is placed at the same level as ind1_c_impressions which are to be aggregated prior to the execution of the script in ind1_ctr, but this led to some other errors.
If someone could point me into the right direction that would be great,
Thanks in advance.