Pipeline aggregation throwing an error

[saurab (Saurab Marjara)]

I am working with the kibana_sample_data_ecommerce index. I will display two queries here:

GET kibana_sample_data_ecommerce/_search?size=0
{
  "aggs": {
    "customer_centric": {
      "terms": {
        "field": "customer_full_name.keyword", 
        "size": 5000
      },
      "aggs": {
        "myagg": {
          "terms": {
            "field": "category.keyword"
          }, 
          "aggs": {
            "taxful_total_price_for_this_category": {
              "sum": {
                "field": "taxful_total_price"
              }
            }, 
            "number_of_orders_for_this_category": {
              "cardinality": {
                "field": "order_id"
              }
            }
          }
        }
      }
    }
  }
}

The result is:

"aggregations" : {
    "customer_centric" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "Elyssa Hansen",
          "doc_count" : 7,
          "myagg" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "Women's Clothing",
                "doc_count" : 5,
                "taxful_total_price_for_this_category" : {
                  "value" : 234.84375
                },
                "number_of_orders_for_this_category" : {
                  "value" : 5
                }
              },
              {
                "key" : "Women's Accessories",
                "doc_count" : 4,
                "taxful_total_price_for_this_category" : {
                  "value" : 280.9375
                },
                "number_of_orders_for_this_category" : {
                  "value" : 4
                }
              },
              {
                "key" : "Women's Shoes",
                "doc_count" : 4,
                "taxful_total_price_for_this_category" : {
                  "value" : 284.921875
                },
                "number_of_orders_for_this_category" : {
                  "value" : 4
                }
              }
            ]
          }
        },

In addition to calculating the number of orders for each category for each customer, I also want to calculate the total number of orders for each customer. So I ran this query:

GET kibana_sample_data_ecommerce/_search?size=0
{
  "aggs": {
    "customer_centric": {
      "terms": {
        "field": "customer_full_name.keyword", 
        "size": 5000
      }, 
      "aggs": {
        "total_number_of_orders": {
          "cardinality": {
            "field": "order_id"
          }
        }
      }, 
      "aggs": {
        "myagg": {
          "terms": {
            "field": "category.keyword"
          }, 
          "aggs": {
            "taxful_total_price_for_this_category": {
              "sum": {
                "field": "taxful_total_price"
              }
            }, 
            "number_of_orders_for_this_category": {
              "cardinality": {
                "field": "order_id"
              }
            }
          }
        }
      }
    }
  }
}

This gave me a compile-time error as follows:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "json_parse_exception",
        "reason" : "Duplicate field 'aggs'\n at [Source: (org.elasticsearch.common.io.stream.ByteBufferStreamInput); line: 15, column: 13]"
      }
    ],
    "type" : "json_parse_exception",
    "reason" : "Duplicate field 'aggs'\n at [Source: (org.elasticsearch.common.io.stream.ByteBufferStreamInput); line: 15, column: 13]"
  },
  "status" : 400
}

It seems that you cannot have two sub-aggregations at the same level for a parent aggregation. How do I modify this query for it to work ? Please help! Thanks in advance.

[asupranovich (Alexander Supranovich)]

Hi @saurab
Try this query:

GET kibana_sample_data_ecommerce/_search?size=0
{
  "aggs": {
    "customer_centric": {
      "terms": {
        "field": "customer_full_name.keyword", 
        "size": 5000
      }, 
      "aggs": {
        "total_number_of_orders": {
          "cardinality": {
            "field": "order_id"
          }
        },
        "myagg": {
          "terms": {
            "field": "category.keyword"
          }, 
          "aggs": {
            "taxful_total_price_for_this_category": {
              "sum": {
                "field": "taxful_total_price"
              }
            }, 
            "number_of_orders_for_this_category": {
              "cardinality": {
                "field": "order_id"
              }
            }
          }
        }
      }
    }
  }
}

[saurab (Saurab Marjara)]

I found out the difficulty. you need to use a sum_bucket aggregation. Here is what works:

GET kibana_sample_data_ecommerce/_search?size=0
{
  "aggs": {
    "customer_centric": {
      "terms": {
        "field": "customer_full_name.keyword", 
        "size": 5000
      }, 
      "aggs": {
        "unique_categories": {
          "terms": {
            "field": "category.keyword"
          }, 
          "aggs": {
            "number_of_orders_for_this_category": {
              "cardinality": {
                "field": "order_id"
              }
            }
          }
        },
        "total_number_of_orders": {
          "sum_bucket": {
            "buckets_path": "unique_categories>number_of_orders_for_this_category"
          }
        }
      }
    }
  }
}

Thanks for the effort.