Is it possible to add aggregations piecemeal to the Nest .NET Library?

Elastic Stack Elasticsearch
1

I have the following code which works:

var search = new SearchDescriptor<Foo>();

search.Aggregations(agg =>
  agg.Cardinality("dedup", cd => cd.Field(fi => fi.AddressHash).PrecisionThreshold(40000))
  .Filter("dontDedup", fl => fl.Filter(ft => ft.Term(te => te.AddressHash, "0")))
  .Min("min_lat", fl => fl.Field(fi => fi.Latitude))
  .Max("max_lat", fl => fl.Field(fi => fi.Latitude))
  .Min("min_lon", fl => fl.Field(fi => fi.Longitude))
  .Max("max_lon", fl => fl.Field(fi => fi.Longitude))
);

But what I want is to add the aggregations to the SearchDescriptor piecemeal.

search.Aggregations(agg =>
  agg.Cardinality("dedup", cd => cd.Field(fi => fi.AddressHash).PrecisionThreshold(40000))
  .Filter("dontDedup", fl => fl.Filter(ft => ft.Term(te => te.AddressHash, "0"))));

then, if a condition is met

if (someCondition) {
    search.Aggregations(agg => agg
        .Min("min_lat", fl => fl.Field(fi => fi.Latitude))
        .Max("max_lat", fl => fl.Field(fi => fi.Latitude))
        .Min("min_lon", fl => fl.Field(fi => fi.Longitude))
        .Max("max_lon", fl => fl.Field(fi => fi.Longitude))
    );
}

However, the 2nd search.Aggregations... wipes out the aggregations added in the 1st statement.
Is it possible to add .Aggregations piecemeal? I didn't see a .Add method or anything like that.

2

There are a couple of approaches typically taken

  1. search.Aggregations(...) takes a Func<AggregationContainerDescriptor<T>, IAggregationContainer>, or in words, a method that accepts an AggregationContainerDescriptor<T> and returns an IAggregationContainer.

    Rather than passing a lambda expression as a delegate inline, you could pull this out into a method that accepts AggregationContainerDescriptor<T> and returns IAggregationContainer. Any logic for adding aggregations can be implemented in that method.

or

  1. Use the object initializer syntax to compose an IAggregationContainer, using an IDictionary<string, IAggregate> and AggregationDictionary, or using overloaded operators on IAggregate types to && them together.

Take a look at the Writing Aggregations documentation for some examples.

3

@forloop I am still not grokking how to actually accomplish this. In the link you provided, there are references to the Combine method, but it's not shown how to implement it.

Using my example, can you show how to accomplish this?

4

Here's an example. Starting with converting the lamdba expression to one with a statement body

var client = new ElasticClient();

var includeMinMax = true;

client.Search<MyDocument>(s => s
    .Aggregations(agg =>
    {
        agg.Cardinality("dedup", cd => cd
            .Field(fi => fi.AddressHash)
            .PrecisionThreshold(40000)
        )
        .Filter("dontDedup", fl => fl
            .Filter(ft => ft
                .Term(te => te.AddressHash, "0")
            )
        );

        if (includeMinMax)
        {
            agg.Min("min_lat", fl => fl.Field(fi => fi.Latitude))
               .Max("max_lat", fl => fl.Field(fi => fi.Latitude))
               .Min("min_lon", fl => fl.Field(fi => fi.Longitude))
               .Max("max_lon", fl => fl.Field(fi => fi.Longitude));
        }

        return agg;
    })
);

You can then extract the lambda statement body out to a method

client.Search<MyDocument>(s => s
    .Aggregations(agg => BuildAggregations(agg, includeMinMax))
);

private static IAggregationContainer BuildAggregations(AggregationContainerDescriptor<MyDocument> agg, bool includeMinMax)
{
    agg.Cardinality("dedup", cd => cd
        .Field(fi => fi.AddressHash)
        .PrecisionThreshold(40000)
    )
    .Filter("dontDedup", fl => fl
        .Filter(ft => ft
            .Term(te => te.AddressHash, "0")
        )
    );
    
    if (includeMinMax)
    {
        agg.Min("min_lat", fl => fl.Field(fi => fi.Latitude))
           .Max("max_lat", fl => fl.Field(fi => fi.Latitude))
           .Min("min_lon", fl => fl.Field(fi => fi.Longitude))
           .Max("max_lon", fl => fl.Field(fi => fi.Longitude));
    }

    return agg;
}

Or even the entire lambda

client.Search<MyDocument>(s => s
    .Aggregations(BuildAggregations(includeMinMax))
);

private static Func<AggregationContainerDescriptor<MyDocument>, IAggregationContainer> BuildAggregations(bool includeMinMax)
{
    return agg =>
    {
        agg.Cardinality("dedup", cd => cd
            .Field(fi => fi.AddressHash)
            .PrecisionThreshold(40000)
        )
        .Filter("dontDedup", fl => fl
            .Filter(ft => ft
                .Term(te => te.AddressHash, "0")
            )
        );

        if (includeMinMax)
        {
            agg.Min("min_lat", fl => fl.Field(fi => fi.Latitude))
               .Max("max_lat", fl => fl.Field(fi => fi.Latitude))
               .Min("min_lon", fl => fl.Field(fi => fi.Longitude))
               .Max("max_lon", fl => fl.Field(fi => fi.Longitude));
        }

        return agg;
    };
}

For the object initializer syntax

var client = new ElasticClient();
var includeMinMax = true;

var aggs = new Dictionary<string, AggregationContainer>{
    { "dedup", new CardinalityAggregation("dedup", Infer.Field<MyDocument>(f => f.AddressHash))
                {
                    PrecisionThreshold = 40000
                }
    },
    { "filter", new FilterAggregation("dontDedup")
                {
                    Filter = new TermQuery
                    {
                        Field = Infer.Field<MyDocument>(f => f.AddressHash),
                        Value = "0"
                    }
                }
    }
};

if (includeMinMax) 
{
    aggs.Add(...);
}

client.Search<MyDocument>(new SearchRequest<MyDocument>
{
   Aggregations = new AggregationDictionary(aggs)
});

Using a dictionary can be long winded, so aggregations can be combined with &&

var client = new ElasticClient();
var includeMinMax = true;

AggregationBase aggs = 
    new CardinalityAggregation("dedup", Infer.Field<MyDocument>(f => f.AddressHash))
    {
        PrecisionThreshold = 40000
    };

aggs &= new FilterAggregation("dontDedup")
    {
        Filter = new TermQuery
        {
            Field = Infer.Field<MyDocument>(f => f.AddressHash),
            Value = "0"
        }
    };

if (includeMinMax) 
{
    // && additional aggs
}

client.Search<MyDocument>(new SearchRequest<MyDocument>
{
   Aggregations = aggs
});
5

Thank you for the detailed answer! I guess I didn't understand lambdas as well as I thought I did.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.