# Scroll & heap size\\ another memory configuration **URL:** \ **Category:** Elasticsearch\ **Created:** [July 17, 2020, 11:03pm UTC](https://discuss.elastic.co/t/scroll-heap-size-another-memory-configuration/241732 "2020-07-17T23:03:18Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![liron\_gofberg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/liron_gofberg/32/59823_2.png) [@liron\_gofberg](https://discuss.elastic.co/u/liron_gofberg)\ **Post date:** [July 17, 2020, 11:03pm UTC](https://discuss.elastic.co/t/scroll-heap-size-another-memory-configuration/241732/1 "2020-07-17T23:03:18Z")
Hi, I want to fetch 50 millions of records into data frame with scan(scroll...) what heap size do I need for that procedure? How can I know what is the right size for me? do I need to configure it only in the master node? In all the nodes? In the client side? there is something else that I need to configure to make this procedure running without any problems.? please.. thanks, Liron ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 21, 2020, 12:20am UTC](https://discuss.elastic.co/t/scroll-heap-size-another-memory-configuration/241732/2 "2020-07-21T00:20:12Z")
What problems are you having? How big are each scroll sections you are requesting? ---
**Author:** ![liron\_gofberg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/liron_gofberg/32/59823_2.png) [@liron\_gofberg](https://discuss.elastic.co/u/liron_gofberg)\ **Post date:** [July 21, 2020, 5:27am UTC](https://discuss.elastic.co/t/scroll-heap-size-another-memory-configuration/241732/3 "2020-07-21T05:27:47Z")
batch size is 1000 and I’ve tried to scroll 11Millions records smaller queries works fine I have 3 nodes heap size at each node was 1.9GB I changed it to 8GB I query with routing value so I query from one shard only Do I need to increase the heap size? what else but heap size can cause memory issue? Thanks!! ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 21, 2020, 5:32am UTC](https://discuss.elastic.co/t/scroll-heap-size-another-memory-configuration/241732/4 "2020-07-21T05:32:25Z")
How big is the total amount of data? ---
**Author:** ![liron\_gofberg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/liron_gofberg/32/59823_2.png) [@liron\_gofberg](https://discuss.elastic.co/u/liron_gofberg)\ **Post date:** [July 21, 2020, 6:46am UTC](https://discuss.elastic.co/t/scroll-heap-size-another-memory-configuration/241732/5 "2020-07-21T06:46:28Z")
Now the total data is 20 Millions records(2 shards) with routing value I query 11M records out of 11M records from one of the two shards ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 21, 2020, 11:45pm UTC](https://discuss.elastic.co/t/scroll-heap-size-another-memory-configuration/241732/6 "2020-07-21T23:45:12Z")
How big in GB. ---
**Author:** ![liron\_gofberg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/liron_gofberg/32/59823_2.png) [@liron\_gofberg](https://discuss.elastic.co/u/liron_gofberg)\ **Post date:** [July 22, 2020, 5:57am UTC](https://discuss.elastic.co/t/scroll-heap-size-another-memory-configuration/241732/7 "2020-07-22T05:57:44Z")
1GB - 5M documents ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 19, 2020, 5:57am UTC](https://discuss.elastic.co/t/scroll-heap-size-another-memory-configuration/241732/8 "2020-08-19T05:57:51Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.