NestedAggregation in co.elastic.clients:elasticsearch-java

Elastic Stack Elasticsearch
1

Hi,

I'm currently migrating my project to use the new client co.elastic.clients:elasticsearch-java:7.17.22. Previously, with org.elasticsearch:elasticsearch:7.8.1, I created nested aggregations in my JUnit tests like this:

TermsAggregationBuilder expectedTerms = new TermsAggregationBuilder("agg");
        expectedTerms.field("data.Nested.NestedInner.NestedInnerField");
        expectedTerms.size(1000);
NestedAggregationBuilder expectedInnerNested = new NestedAggregationBuilder("nested", "data.Nested.NestedInner");
expectedInnerNested.subAggregation(expectedTerms);

NestedAggregationBuilder expectedParentNested = new NestedAggregationBuilder("nested", "data.Nested");
expectedParentNested.subAggregation(expectedInnerNested);

This would return a JSON structure like:

{
   "nested": {
      "nested": {
         "path": "data.Nested"
      },
      "aggregations": {
         "nested": {
            "nested": {
               "path": "data.Nested.NestedInner"
            },
            "aggregations": {
               "agg": {
                  "terms": {
                     "field": "data.Nested.NestedInner.NestedInnerField",
                     "size": 1000,
                     "min_doc_count": 1,
                     "shard_min_doc_count": 0,
                     "show_term_doc_count_error": false,
                     "order": [
                        { "_count": "desc" },
                        { "_key": "asc" }
                     ]
                  }
               }
            }
         }
      }
   }
}

Now, with the new client, I tried to create the same nested aggregation using:

String jsonString = "{\"nested\":{\"nested\":{\"path\":\"data.Nested\"},\"aggregations\":{\"nested\":{\"nested\":{\"path\":\"data.Nested.NestedInner\"},\"aggregations\":{\"agg\":{\"terms\":{\"field\":\"data.Nested.NestedInner.NestedInnerField\",\"size\":1000,\"min_doc_count\":1,\"shard_min_doc_count\":0,\"show_term_doc_count_error\":false,\"order\":[{\"_count\":\"desc\"},{\"_key\":\"asc\"}]}}}}}}}";

return new NestedAggregation.Builder()
    .withJson(stringReader)
    .build();

or

new Aggregation
            .Builder()
                .withJson(stringReader)
                .build();

However, I received the following error:

co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch._types.aggregations.NestedAggregation: Unknown field 'nested' (JSON path: nested) (line no=1, column no=10, offset=9)

or

Caused by: co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch._types.aggregations.NestedAggregation: Unknown field 'nested' (JSON path: nested.nested) (line no=1, column no=20, offset=19)
	at co.elastic.clients.json.ObjectDeserializer.parseUnknownField

Can anyone help me understand what I'm doing wrong? How can I properly create a NestedAggregation with sub-aggregations using the new client?

Thanks!

2

Hello and welcome!

The first part of the json structure you sent:

   "nested": {
      "nested": {
         "path": "data.Nested"
      },

isn't part of the aggregation structure, that's why the new client doesn't recognize it. I was trying to understand what it actually is by sending the request to the server using the Kibana dev tools, but it doesn't seem to be a query, could you give me more information on this?

In the meantime, this is how to write a nested aggregation with the 7.17.21 version of the java client:

 esClient.search(s -> s.index("my-index")
     .aggregations("nested", a -> a
         .nested(n -> n.path("data.Nested.NestedInner"))
         .aggregations("agg", ag -> ag
             .terms(t -> t
                 .field("data.Nested.NestedInner.NestedInnerField")
                 .size(1000)
                 .minDocCount(1)
                 .showTermDocCountError(false)
                  .order(NamedValue.of("_count", SortOrder.Desc),NamedValue.of("_key",SortOrder.Asc)))))
     , Object.class);

There's one field missing which is shard_min_doc_count, we just recently noticed it was missing from TermsAggregation, we already merged a fix for it which will be available in the 7.17.23 version coming out in a few days.

3

Hi,

We have a junit test to create nestedAggregations + subAggregations:

TermsAggregationBuilder expectedTerms = new TermsAggregationBuilder("agg");
        expectedTerms.field("data.Nested.NestedInner.NestedInnerField");
        expectedTerms.size(1000);
        NestedAggregationBuilder expectedInnerNested = new NestedAggregationBuilder("nested", "data.Nested.NestedInner");
        expectedInnerNested.subAggregation(expectedTerms);
        NestedAggregationBuilder expectedParentNested = new NestedAggregationBuilder("nested1", "data.Nested");
        expectedParentNested.subAggregation(expectedInnerNested);

expectedParentNested will have

{
   "nested":{
      "nested":{
         "path":"data.Nested"
      },
      "aggregations":{
         "nested":{
            "nested":{
               "path":"data.Nested.NestedInner"
            },
            "aggregations":{
               "agg":{
                  "terms":{
                     "field":"data.Nested.NestedInner.NestedInnerField",
                     "size":1000,
                     "min_doc_count":1,
                     "shard_min_doc_count":0,
                     "show_term_doc_count_error":false,
                     "order":[
                        {
                           "_count":"desc"
                        },
                        {
                           "_key":"asc"
                        }
                     ]
                  }
               }
            }
         }
      }
   }
}

As far as I understand this test, we wanted to have

"nested":{
         "path":"data.Nested",
         "name": "nested"
      }

instead of

 "nested":{
      "nested":{
         "path":"data.Nested"
      }

In java docs, I found `NestedAggregationBuilder(name, path), name - is name of Aggregation but not a node 'Nested'

A new client:

Aggregation.of(builder -> builder
                .nested(n -> n.path("data.Nested"))
                .aggregations("nested", n -> n.nested(s ->s.path("data.Nested.NestedInner").name("nested1"))).aggregations("agg",
                        a -> a.terms(t -> t.field("data.Nested.NestedInner.NestedInnerField").size(100))));

nested:{
"path": data.Nested.NestedInner
"name: nested1
}

I think it is OK.

4

Hope it helped! If you're still having issues with aggregations here are some other examples of nested/sub aggregations:

1 Like
5

Thank you :wink: