# Rally dump existing cluster data while 429 occur

**URL:** https://discuss.elastic.co/t/rally-dump-existing-cluster-data-while-429-occur/275385
**Category:** Elasticsearch
**Tags:** rally
**Created:** [June 9, 2021, 7:29am UTC](https://discuss.elastic.co/t/rally-dump-existing-cluster-data-while-429-occur/275385 "2021-06-09T07:29:57Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![JohnPhilip](https://avatars.discourse-cdn.com/v4/letter/j/57b2e6/32.png) [@JohnPhilip](https://discuss.elastic.co/u/JohnPhilip)
#### Post date: [June 9, 2021, 7:29am UTC](https://discuss.elastic.co/t/rally-dump-existing-cluster-data-while-429-occur/275385/1 "2021-06-09T07:29:57Z")

</div>

when I try to [create a track from data in an existing cluster](https://esrally.readthedocs.io/en/latest/adding_tracks.html#creating-a-track-from-data-in-an-existing-cluster), I found that error occurred like following

```auto
Extracting documents for index [compress_ratio_ac... 1650000/50000001 docs [3.3% done][ERROR] Cannot create-track. TransportError(429, 'circuit_breaking_exception', '[parent] Data too large, data for [<http_request>] would be [1702314100/1.5gb], which is larger than the limit of [1690094796/1.5gb], real usage: [1702313936/1.5gb], new bytes reserved: [164/164b], usages [request=0/0b, fielddata=0/0b, in_flight_requests=154112090/146.9mb, accounting=0/0b]').

```

how can I solve this problem if I don't set ES config **indices.fielddata.cache.size** , rally can do something to avoid this situation?

---

<div class="post-metadata">

### Author: ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)
#### Post date: [June 9, 2021, 8:58am UTC](https://discuss.elastic.co/t/rally-dump-existing-cluster-data-while-429-occur/275385/2 "2021-06-09T08:58:36Z")

</div>

Hi,

extracting all data from a cluster puts a heavier burden than usual operation on the cluster. Looking at the circuit breaker exception I guess that you limit heap size to ~ 2GB. I suggest you temporarily allocate more heap memory to Elasticsearch when extracting data.

Daniel

---

<div class="post-metadata">

### Author: ![JohnPhilip](https://avatars.discourse-cdn.com/v4/letter/j/57b2e6/32.png) [@JohnPhilip](https://discuss.elastic.co/u/JohnPhilip)
#### Post date: [June 9, 2021, 9:29am UTC](https://discuss.elastic.co/t/rally-dump-existing-cluster-data-while-429-occur/275385/3 "2021-06-09T09:29:09Z")

</div>

thanks Daniel,  
maybe the cluster has other pressure of writing data when I extracting data from it, which cause the JVM memory to be insufficient, i'll try later, but there is no way to solve it by configuring rally right? perhaps only configure ES can avoid this if i have limited JVM memory

Dzp

---

<div class="post-metadata">

### Author: ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)
#### Post date: [June 9, 2021, 11:30am UTC](https://discuss.elastic.co/t/rally-dump-existing-cluster-data-while-429-occur/275385/4 "2021-06-09T11:30:38Z")

</div>

Hi,

no, it's unfortunately not configurable. Internally, Rally uses the Python client's [scan helper](https://elasticsearch-py.readthedocs.io/en/v7.13.1/helpers.html#scan) which fetches documents in batches of size 1000.

Daniel

---

<div class="post-metadata">

### Author: ![JohnPhilip](https://avatars.discourse-cdn.com/v4/letter/j/57b2e6/32.png) [@JohnPhilip](https://discuss.elastic.co/u/JohnPhilip)
#### Post date: [June 9, 2021, 1:06pm UTC](https://discuss.elastic.co/t/rally-dump-existing-cluster-data-while-429-occur/275385/5 "2021-06-09T13:06:54Z")

</div>

thanks again Daniel,

it's so helpful to me, glad to talk with you

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 7, 2021, 1:06pm UTC](https://discuss.elastic.co/t/rally-dump-existing-cluster-data-while-429-occur/275385/6 "2021-07-07T13:06:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
