# Search scroll high off-heap memory usage

**URL:** <https://discuss.elastic.co/t/search-scroll-high-off-heap-memory-usage/300812>\
**Category:** Elasticsearch\
**Tags:** docker\
**Created:** [March 28, 2022, 8:34am UTC](https://discuss.elastic.co/t/search-scroll-high-off-heap-memory-usage/300812 "2022-03-28T08:34:58Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\
**Post date:** [March 28, 2022, 8:34am UTC](https://discuss.elastic.co/t/search-scroll-high-off-heap-memory-usage/300812/1 "2022-03-28T08:34:58Z")

</div>

**Cluster:**

- Elasticsearch v7.17.1 running with elastic docker image on kubernetes
- 6 data nodes: 4 CPUs / 14 Go HEAP / 28 Go memory
- Indice: 3 primary shards (+ 1 replica) , 60 Go data

**Usage:**

- nightly full index export, with Apache Spark using elastic for hadoop lib
- scroll size: 1200
- 6 tasks in //

**Monitoring:**

 ![Capture d’écran 2022-03-28 à 10.29.28](https://us1.discourse-cdn.com/elastic/original/3X/0/5/0534cdb3b96fc5d8bff2cd0bc340d8ca67abb632.jpeg)

 ![Capture d’écran 2022-03-28 à 10.29.03](https://us1.discourse-cdn.com/elastic/original/3X/2/6/26d85a7774a2a4637fc119ca096f42e4baa3c191.jpeg)

 ![Capture d’écran 2022-03-28 à 10.30.06](https://us1.discourse-cdn.com/elastic/original/3X/d/8/d84a747d22fe625395fa6d2ef7746adc2f68ed6d.jpeg)

**Question:**

All data nodes consume +10Go memory during the long search scroll phase, the heap seems fine, how can I debug what is in this memory please?

This is a problem for us because Spark is running on the same k8s cluster and consume also lot of memory.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 28, 2022, 8:40am UTC](https://discuss.elastic.co/t/search-scroll-high-off-heap-memory-usage/300812/2 "2022-03-28T08:40:53Z")

</div>

It's likely the OS caching the underlying files that are being requested by Elasticsearch.

---

<div class="post-metadata">

**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\
**Post date:** [March 28, 2022, 8:49am UTC](https://discuss.elastic.co/t/search-scroll-high-off-heap-memory-usage/300812/3 "2022-03-28T08:49:28Z")

</div>

Thanks you, I will update our Memory Usage graphs !

BTW, is off-heap Elasticsearch usage reported somewhere?

---

<div class="post-metadata">

**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\
**Post date:** [March 28, 2022, 1:48pm UTC](https://discuss.elastic.co/t/search-scroll-high-off-heap-memory-usage/300812/4 "2022-03-28T13:48:30Z")

</div>

Well I am not sure, our infra guys told me page cache is not reported as WSS or RSS, I can graph it and I can see this is not related

 ![Capture d’écran 2022-03-28 à 15.47.18](https://us1.discourse-cdn.com/elastic/original/3X/b/b/bb68f846081acf35312d57653cbb3606469ce4b8.jpeg)

- blue = cache
- yellow = RSS

So the truth is out there 😄

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 25, 2022, 1:48pm UTC](https://discuss.elastic.co/t/search-scroll-high-off-heap-memory-usage/300812/5 "2022-04-25T13:48:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.