High Heap Usage

[yasir_khatri]

We are using single node of elasticsearch v7.9.1 with 3.0 Tb of data. The rollover size is 10 gb 3 shards per index, that makes in 3.33 gb per shard.
I need help troubleshooting these issues:

• The heap usage is always around 85%, though the heap size is 20gigs.
• We are constantly getting Data too large exception, when indexing more data.

My guess is the rollover size is too small which is creating too many small shards and causing high heap usage and thats tripping the circuit breaker

Please Advice

Thanks and Regrads

[warkolm (Mark Walkom)]

Welcome to our community!

@yasir_khatri

The rollover size is 10 gb 3 shards per index, that makes in 3.33 gb per shard

That's pretty inefficient. I would suggest you increase that and shrink some of your indices.

[yasir_khatri]

Thanks for replaying, any suggestion what max_size should i use for rollover?
Regards,

[warkolm (Mark Walkom)]

We normally suggest in the range of 30-50GB.