# Composite Aggregation Sorting Based on Field Which is Not Key

**URL:** <https://discuss.elastic.co/t/composite-aggregation-sorting-based-on-field-which-is-not-key/248146>\
**Category:** Elasticsearch\
**Created:** [September 10, 2020, 10:31am UTC](https://discuss.elastic.co/t/composite-aggregation-sorting-based-on-field-which-is-not-key/248146 "2020-09-10T10:31:54Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mstzn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mstzn/32/56322_2.png) [@mstzn](https://discuss.elastic.co/u/mstzn)\
**Post date:** [September 10, 2020, 10:31am UTC](https://discuss.elastic.co/t/composite-aggregation-sorting-based-on-field-which-is-not-key/248146/1 "2020-09-10T10:31:54Z")

</div>

Hello Everyone,

I have a composite aggregation which aggregate bucket based on a term field like number field. Composite aggregation orders buckets by natural order of key.  
I want to order buckets based on max date on a field.

In the below code block i dont want to order based on fromNumber order. I have a startDate field in a document. And I want to sort buckets by up-to-date of startDate field.

I there any possibility ?

```auto
      "composite": {
        "size": 20,
        "sources": [
    
          {
            "byFromNumber": {
              "terms": {
                "field": "fromNumber",
                "missing_bucket": false,
                "order":"asc"
              }
            }
          }
        ],
        "after": {
          "byFromNumber": ""
        }
      },

```

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [September 10, 2020, 11:22am UTC](https://discuss.elastic.co/t/composite-aggregation-sorting-based-on-field-which-is-not-key/248146/2 "2020-09-10T11:22:09Z")

</div>

It's one of those "it depends" answers I'm afraid.

How many shards/indices do you have?  
How many unique numbers do you group on?

In a distributed system the constraints of how much data you can carry back from each shard make it complex - the same way the fox, the chicken and the grain problem is complicated by the constraint of a small boat.  
This [wizard](https://plnkr.co/edit/eZr7r3KZW02AxNKAHCQa?p=preview&preview) walks through some of the options.

---

<div class="post-metadata">

**Author:** ![mstzn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mstzn/32/56322_2.png) [@mstzn](https://discuss.elastic.co/u/mstzn)\
**Post date:** [September 10, 2020, 12:00pm UTC](https://discuss.elastic.co/t/composite-aggregation-sorting-based-on-field-which-is-not-key/248146/3 "2020-09-10T12:00:25Z")

</div>

Thank you for response @Mark_Harwood

I have 1 indices and 3 shard . Unique numbers can be 500k. Here is my full es request. I just simple want to get first 20 bucket order by callStartStamp desc

```auto
{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "studenId": {
              "value": 8525,
              "boost": 1
            }
          }
        }
      ],
      "adjust_pure_negative": true,
      "boost": 1
    }
  },
  "sort": [
    {
      "callStartStamp": {
        "order": "desc"
      }
    }
  ],
  "aggregations": {
    "fromNumberAgg": {
      "composite": {
        "size": 20,
        "sources": [
          {
            "byFromNumber": {
              "terms": {
                "field": "fromNumber",
                "missing_bucket": false,
                "order": "asc"
              }
            }
          }
        ],
        "after": {
          "byFromNumber": ""
        }
      },
      "aggregations": {
        "hits": {
          "top_hits": {
            "from": 0,
            "size": 1000,
            "version": false,
            "seq_no_primary_term": false,
            "explain": false,
            "sort": [
              {
                "callStartStamp": {
                  "order": "desc"
                }
              }
            ]
          }
        },
        "lastCallStart": {
          "max": {
            "field": "callStartStamp"
          }
        },
        "sortByCallStart": {
          "bucket_sort": {
            "sort": [
              {
                "lastCallStart": {
                  "order": "desc"
                }
              }
            ],
            "from": 0,
            "gap_policy": "SKIP"
          }
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [September 10, 2020, 1:07pm UTC](https://discuss.elastic.co/t/composite-aggregation-sorting-based-on-field-which-is-not-key/248146/4 "2020-09-10T13:07:17Z")

</div>

Looks like you want 1,000 records for each of those top 20.  
However, in order to ensure accuracy of results in `terms` aggregations (the one you want to group on) we ask for more than 20 results from each shard. These are promising candidates for the final cut and only by merging results from multiple shards do we get towards an accurate picture of each candidate. So we throw away a lot of candiates in the final fusion - including all their \<=1000 top\_hits that may accompany them.  
For this reason it would be better to split this into 2 queries - one to get the 20 most-recently-active callers and then a follow-up request to get just their call histories.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 8, 2020, 1:07pm UTC](https://discuss.elastic.co/t/composite-aggregation-sorting-based-on-field-which-is-not-key/248146/5 "2020-10-08T13:07:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.