# Python Elasticsearch API: Error 'data too large' when iterating

**URL:** <https://discuss.elastic.co/t/python-elasticsearch-api-error-data-too-large-when-iterating/336234>\
**Category:** Elasticsearch\
**Created:** [June 16, 2023, 2:15pm UTC](https://discuss.elastic.co/t/python-elasticsearch-api-error-data-too-large-when-iterating/336234 "2023-06-16T14:15:27Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![tmslara.a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tmslara.a/32/116339_2.png) [@tmslara.a](https://discuss.elastic.co/u/tmslara.a)\
**Post date:** [June 16, 2023, 2:15pm UTC](https://discuss.elastic.co/t/python-elasticsearch-api-error-data-too-large-when-iterating/336234/1 "2023-06-16T14:15:28Z")

</div>

I am using the Python Elasticsearch API to interact with my Elastic cluster. I'm getting an error when I try to perform several searches in a for loop. In synthesis, I'm doing the following: I iterate over a list of values. In each iteration, I take a value from the list and formulate a query with it to retrieve a specific set of documents. Using this query I perform a search (client.search(index, query, aggs)) to get a terms aggregation. After some number of iterations, I receive an error of 'data too large'. I'm confident that the result of each aggregation will not retrieve an excessively large number of terms.

I assume there is some kind of buffer that is not emptied after each iteration. I'd appreciate some insights in this matter.

I include the complete error that I get:

BadRequestError(400, 'search\_phase\_execution\_exception', 'task cancelled [Fatal failure during search: failed to merge result [[parent] Data too large, data for [\<reduce\_aggs\>] would be [999285563/952.9mb], which is larger than the limit of [996147200/950mb], real usage: [999285528/952.9mb], new bytes reserved: [35/35b], usages [inflight\_requests=952/952b, request=106981446/102mb, fielddata=299829275/285.9mb, eql\_sequence=0/0b, model\_inference=0/0b]]]')

I've got the same error before when also using for loops. In those cases I partially solved the issue including the creation of the client inside the loop, however, I can't do that in the case that I previously described (also I think is not desirable). I also get an ApiError(429 sometimes)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 14, 2023, 2:15pm UTC](https://discuss.elastic.co/t/python-elasticsearch-api-error-data-too-large-when-iterating/336234/2 "2023-07-14T14:15:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.