# Fastest way to retrieve all unique terms in batches

**URL:** https://discuss.elastic.co/t/fastest-way-to-retrieve-all-unique-terms-in-batches/385057
**Category:** Elasticsearch
**Created:** [February 15, 2026, 3:03pm UTC](https://discuss.elastic.co/t/fastest-way-to-retrieve-all-unique-terms-in-batches/385057 "2026-02-15T15:03:29Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![srishtii](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/srishtii/32/141029_2.png) [@srishtii](https://discuss.elastic.co/u/srishtii)
#### Post date: [February 15, 2026, 3:03pm UTC](https://discuss.elastic.co/t/fastest-way-to-retrieve-all-unique-terms-in-batches/385057/1 "2026-02-15T15:03:29Z")

</div>

I’m looking for the **fastest and safest way to retrieve _all_ unique values of a field for a given time range** , in batches.

My requirements are:

- ✅ Retrieve **100% of unique terms** (no missing buckets)

- ✅ Support **batching / pagination**

- ✅ Be **as fast as possible**

- ✅ Avoid excessive heap usage

I’ve evaluated two approaches:

* * *

### 1. `terms` aggregation with `include.partition`

Example:

```auto
"terms": {
  "field": "someField.keyword",
  "size": 10000,
  "include": {
    "partition": 0,
    "num_partitions": 20
  }
}

```

Iterating `partition = 0..N`.

I understand that:

- Each unique term is deterministically hashed into a partition

- Distribution may appear uneven for small cardinalities

- Larger cardinalities should distribute more evenly

- The same term always maps to the same partition across shards

However, this approach requires:

- Choosing `num_partitions` upfront

- Managing a hard `size` limit per partition (risk of missing terms)

- Manual orchestration of partitions

- No cursor/resume mechanism

- Potentially higher heap usage due to in-memory bucket building

* * *

### 2. Composite aggregation with `after_key`

This seems to offer:

- Cursor-based pagination

- Unlimited buckets

- Natural batching

- Lower memory pressure

- Easy resumability

* * *

### Question

For the general use case:

> **Retrieve all unique field values over a time range, at scale, with batching and maximum performance**

Is **composite aggregation** the recommended production approach over `terms + partition`?

Are there scenarios where `terms + partition` is preferable?

My primary goal is:

👉 _Fast, complete, resumable extraction of unique terms._

Thanks in advance for any guidance.

* * *

---

<div class="post-metadata">

### Author: ![Mark\_Harwood1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood1/32/101255_2.png) [@Mark\_Harwood1](https://discuss.elastic.co/u/Mark_Harwood1)
#### Post date: [February 15, 2026, 9:47pm UTC](https://discuss.elastic.co/t/fastest-way-to-retrieve-all-unique-terms-in-batches/385057/2 "2026-02-15T21:47:17Z")

</div>

Yes, use composite.

Partitioned terms agg is only generally useful if you are sorting by criteria other than the terms themselves eg getting a list of account IDS that have not been used for a long time (sorting by max date for each term, in reverse order)