High ram usage

Elastic Stack Elasticsearch
1

Hello,

after some posts and good answers we optimize our Elasticsearch.
Actually we use:
6 x hot nodes a 32 gb ram / heap space a 16 gb
22 x cold nodes a 16 gb / 8 gb space
We reduce 8 primaries shards to 6 shards

At the moment we see lower shards and less cpu usage. But we thought that less ram usage will be the result. But it raise day by day.

What can we do to reduce memory usage?

Thanks a lot.

2

Which version of Elasticsearch are you using? Are you forcemerging older read-only indices down to a single segment?

How do you do this?

3 

  "name" : "kibana-com-1-rz1",
  "cluster_name" : "elastic-com-c1",
  "cluster_uuid" : "pk3BirT2SB-Z5MAPU5vC8A",
  "version" : {
    "number" : "7.12.0",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "78722783c38caa25a70982b5b042074cde5d3b3a",
    "build_date" : "2021-03-18T06:17:15.410153305Z",
    "build_snapshot" : false,
    "lucene_version" : "8.8.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
4

We reduce the shards by chaging the index lifecycle policies

5

I would recommend you upgrade your cluster to version 8.7 as some significant improvement around heap usage have been made.

Reducing the number of primary shards does not necessarily reduce heap usage a lot. It is, at least on the version you are on, important to also forcemerge down to a single segment.

6

Thanks a lot.
Actually we see shards, heap space and segments are shrinking:

image
image4563×1025 409 KB

As I understood, with less segments we need less memory.
Besides the update we're planning, we didn't try to forcemerge to a single segment.

But which processes need memory besides the heap space?
I see in our dashboard that many metrics declining, but with no impacts on the entire memory usage....

7

Elasticsearch uses the heap, but also some off-heap memory. In addition to the the operating system page cache is essential for performance and it is common to see all memory in use on an Elasticsearch host.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.