# Elasticsearch Report Download of size more than 10000 using aggregation

**URL:** <https://discuss.elastic.co/t/elasticsearch-report-download-of-size-more-than-10000-using-aggregation/321927>\
**Category:** Elasticsearch\
**Created:** [December 23, 2022, 2:56pm UTC](https://discuss.elastic.co/t/elasticsearch-report-download-of-size-more-than-10000-using-aggregation/321927 "2022-12-23T14:56:39Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![PRASHANT\_MEHTA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_mehta/32/101764_2.png) [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\
**Post date:** [December 23, 2022, 2:56pm UTC](https://discuss.elastic.co/t/elasticsearch-report-download-of-size-more-than-10000-using-aggregation/321927/1 "2022-12-23T14:56:40Z")

</div>

Hello All,

I'm facing issue when I'm trying to download data more than 10000 records.  
I've 25000 documents in my elastic index and all these documents are unique(logsatsh doc\_as\_upsert) used.The challenge is elastic restricts to fetch aggregatable data more than 10000  
records.  
Intention is to download whole data that is available as data is unique per document.How can this be achieved using elastic or using java backend logic?  
The reports can be fetched for 3 months ,6 months or year etc.Hence volume of data would be large,Need to generate excel file for reports but unable to get data more than 10000.  
Tried scroll api,but I suppose that don't serve the purpose.Does scroll API works for aggregation data I suppose no.Java backend logic is wriiten to fetch data and download in excel.

How does huge reporting data download works in elastic ?,like reports daily,monthly,yearly etc.  
I see many mention of scroll API but how its implemented exactly not getting it and not sure in my case it would help.

Any suggestion would be helpful.Elk version 7.9.1.

```auto
GET /cis-monitor-campaignreport-*/_search
{
  "size": 0, 
  "aggs": {
    "uniqueIdList": {
      "terms": {
        "field": "campaignreport.uniqueId.keyword"
      }
    }
  }

```

![image](https://us1.discourse-cdn.com/elastic/original/3X/7/c/7c36653d7a992128b8efe7c133906a41bb0c47ba.png)

The below query work fine for data upto 10000,but fails for anything above,In my case 30000 documents size.

```auto
GET /cis-monitor-campaignreport-*/_search
{
  "size": 20000, 
  "aggs": {
    "uniqueIdList": {
      "terms": {
        "field": "campaignreport.uniqueId.keyword"
      }
    }
  }
}

```

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/0/7067a67b9aacb6f8a58ecbabbc54bafcb0e4d4a6.png)

index.max\_result\_window : I would like to avoid this config as its may have impact on cluster.What are the alternatives?

Many Thanx

---

<div class="post-metadata">

**Author:** ![jeeva042](https://avatars.discourse-cdn.com/v4/letter/j/aca169/32.png) [@jeeva042](https://discuss.elastic.co/u/jeeva042)\
**Post date:** [December 23, 2022, 3:47pm UTC](https://discuss.elastic.co/t/elasticsearch-report-download-of-size-more-than-10000-using-aggregation/321927/2 "2022-12-23T15:47:13Z")

</div>

Hello @PRASHANT_MEHTA,  
Well if you dont want to use the index.max\_result\_window option, you may have to try out with [Search after](https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#search-after) or [scroll search results](https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#scroll-search-results)

- the [search after](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-search-after.html) feature to do deep pagination.
- the [Scroll API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html) if you want to extract a resultset to be consumed by another tool later.

Links:  
[Records more than 10,000 with pagination - Elastic Stack / Elasticsearch - Discuss the Elastic Stack](https://discuss.elastic.co/t/records-more-than-10-000-with-pagination/249398/3)

[Scroll search results](https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#scroll-search-results)

[Search after](https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#search-after)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 20, 2023, 3:47pm UTC](https://discuss.elastic.co/t/elasticsearch-report-download-of-size-more-than-10000-using-aggregation/321927/3 "2023-01-20T15:47:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.