Search scroll high off-heap memory usage


  • Elasticsearch v7.17.1 running with elastic docker image on kubernetes
  • 6 data nodes: 4 CPUs / 14 Go HEAP / 28 Go memory
  • Indice: 3 primary shards (+ 1 replica) , 60 Go data


  • nightly full index export, with Apache Spark using elastic for hadoop lib
  • scroll size: 1200
  • 6 tasks in //



All data nodes consume +10Go memory during the long search scroll phase, the heap seems fine, how can I debug what is in this memory please?

This is a problem for us because Spark is running on the same k8s cluster and consume also lot of memory.

It's likely the OS caching the underlying files that are being requested by Elasticsearch.

Thanks you, I will update our Memory Usage graphs !

BTW, is off-heap Elasticsearch usage reported somewhere?

Well I am not sure, our infra guys told me page cache is not reported as WSS or RSS, I can graph it and I can see this is not related

  • blue = cache
  • yellow = RSS

So the truth is out there :smile:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.