Custom aggregation plugin with multiple result reader

Elastic Stack Elasticsearch
1

Hi all.
I am writing a specific aggregation plugin (not the first one I write...)
for ES version 6.6.

This plugin can compute various aggregations (intended for process mining) on the data,
with only one aggregation builder class on input. (We make these computation in a plugin for performance reasons).

My issue is the return type of the aggregation query.
I would like the return type to depend on the aggregation query configuration parameters.

{...,
     "aggs" : { "first" : {"my_custom_aggregation": {"algorithm" : "firstAlgorithm"}},
                "second": {"my_custom_aggregation": {"algorithm" : "secondAlgorithm"}},
                 "third": {"my_custom_aggregation": {"algorithm" : "thirdAlgorithm"}},
     }
 }

For the first algorithm, I return a list of things (so-called "processes") in a
class FirstAlgorithmInternalAggregation extends InternalAggregation,
I do not even need buckets in this case.

For the second algorithm, I return a true list of buckets in a
class SecondAlgorithmInternalAggregation extends InternalMultiBucketAggregation

For the third algorithm, I return a true list of buckets in a
class ThirdAlgorithmInternalAggregation extends InternalMultiBucketAggregation

The buckets in these results are enriched with various information, that vary with the algorithm used.

When I query and get the response through a python script (using python module "elasticsearch"), the answers come back in python dictionaries, and I can parse them, extract the results, etc...
I am guessing the python module goes through the REST API and that the JSON serialization deals with the various result types.

When I query through Java API, I have a problem: from what I understand the serialization of the answer must be known and declared at plugin registration time in the

List<AggregationSpec> getAggregations() {...}

method of the MyCustomPlugin class.
Right now, I register only one AggregationSpec, with one result reader there, which reads only one type of InternalAggregation.

I am seeking to build and return different "InternalAggregation"'s in the response, depending on the plugin parameters.

So my questions:

  1. Is it even possible to return objects of different types through the Java API, based only on the builder parameters?
  2. If I return more than one AggregationSpec from getAggregations, what are the semantics?
  3. Could I design a result reader that would be able to deserialize various return types? Not sure this is possible.

I could probably unify the return types, but they are quite different. I would like to avoid it.

I could also separate the entry points for the three algorithms, but it would be nice to avoid creating a different entry point for each algorithm (the computation machineries have a lot in common under the hood).

Any help appreciated.
Thanks for reading!

Sylvain Mazet.

2

Partially answering myself here.

I opted for multiple entry points,
and partially answered question 2:
returning multiple AggregationSpec allows me to register
more than one builders.

(Partially because I still don't know what happens if I register more than one ResultReader
per AggregationSpec).

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.