What is future plan of scripted aggregations

Elastic Stack Elasticsearch
1

hello team,
We are planning to use scripted aggregations feature of elasticsearch. But are apprehensive about it because doc [ https://www.elastic.co/guide/en/elasticsearch/reference/6.x/search-aggregations-metrics-scripted-metric-aggregation.html ] for it says that it is experimental feature and can be removed in future GA releases.

So coming to my question, is there a word about this feature getting permanent or not.

2

I think at this point we would probably not remove the scripted metric aggregation without a deprecation period and a suitable replacement for the types of analytics users are using it for. The experimental tag still exists because up to this point we have not been completely happy with its API and this is something we are working to solve with changes like https://github.com/elastic/elasticsearch/pull/30111.

I think there is likely to always be a place for writing custom aggregation logic using a script (at least while creating a plugin that adds a new aggregation is still complex) but I would always see it as a last resort. Many times I have seen users reach for the scripted_metric aggregation before fully exploring whether their use case can be solved in a different way with the current aggregations. I am not saying that your case is one of these but I would advise exploring options for not using the scripted_metric aggregation where you can.

Also, as we are always looking to expand the list of out of the box aggregations, if your use case feels like it might be generally useful then feel free to open an issue on the Elasticsearch Github repo and we can explore whether we should build a dedicated aggregation to solve use cases like yours.

1 Like
3

thanks for the heads up and valuable information @colings86 .
Didn't want to stray offtopic but the usecase I'm trying to solve is

currently we sum over single field. but I want to sum up over 1 field and if that field is not present then some another field should be added to the sum.

PS. I always wanted to thank someone from elasticsearch for such a great & heavy duty thing. So "thanks" :slight_smile:

4

So actually this is a use case where I would advise not using the scripted_metric aggregation but instead to use the fact that on most aggregations that you can specify a field you can instead specify a script. See https://www.elastic.co/guide/en/elasticsearch/reference/6.2/search-aggregations-metrics-sum-aggregation.html#_script_9 for more information on this for the sum aggregation.

So for your use case I would use the sum aggregation but instead of specifying the field I would specify a script option something like the following (disclaimer: I have not tested this so the syntax might need a bit of tweaking):

{
	"size": 0,
	"aggs": {
		"my_sum": {
			"sum": {
				"script": {
					"source": "doc[params.primary_field].value == null ? doc[params.secondary_field].value : doc[params.primary_field].value",
					"params": {
						"primary_field": "my_field",
						"secondary_field": "my_field2"
					}
				}
			}
		}
	}
}

That way you don't need to worry about the complexity of the scripted_metric aggregation.

1 Like
5

thank you so much Colin , Thank you so much.

6

doc[params.primary_field].value == null

Just a minor note: this won't work. If the field is missing, .value currently returns a "default" value. There is some work being done to make this better (Handle missing and multiple values in script by mayya-sharipova · Pull Request #29611 · elastic/elasticsearch · GitHub), but in the meantime, you can check the size of the return object from doc.

doc[params.primary_field].size() == 0
1 Like
7

thanks @rjernst , worked like a charm. Thanks for your time.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.