Heap usage in versions 6 and 7

Elastic Stack Elasticsearch
1

After upgrading from 6.7.0 to 7.10.2, the amount of data in each data node has decreased, but the heap utilization rate has increased, leading to frequent circuit_breaking. The 7 version FST has moved to the off heap memory, so the memory utilization rate should be lower？I don't understand. Is there a bug？

2

3.3TB data per data node of the 6.7.0 version
only 1.5TB data per data node of the 7.10.2 version

image
image713×420 67.2 KB

image
The data format and cluster settings are the same, but the heap usage of cluster 7 is higher

3

You can dump the heap to check which class lead to high usage of heap.

5

What is the full output of the cluster stats API?

I would agree with DJ_Zhu and recommend having a look at a heap dump to see what is causing the increase.

6

Hi, we got the same problem. Upgrading from 6.8 to 7.17.8 I can see x3 increase in fixed_bit_set_memory_in_bytes usage. I would assume it's connected to heavy nested field usage in our mapping, but such an increase is a bit strange. We can see a decrease in terms memory usage but it's not enough to offset the 3x increase in fixed bitset.

7

How did you solve it？