Negative Request Breaker Estimated Size

Elastic Stack Elasticsearch
1

Hi Everyone,

I am using ES-8.15.5 and while using this in production I am getting negative value of requests circuit breaker estimated_size_in_bytes.

Not able to reproduce this locally either. I am having hard time identifying the cause for this. Can someone please help on how can I identify the cause for this?

image
image426×186 26.4 KB

I turned on Trace logging for a while and got logs similar to this of addition or subtraction:
[request] Adjusted breaker by [-16472] bytes, now [-9914192]

Let me know if any other detail is required here.

2

This is a definite bug; you’re using a version that’s almost a year old now, can you upgrade and see if it’s been fixed in the meantime?

3

Actually we recently moved from 7.17.6 to 8.15.5, so really can’t move immediately. Is it possible to find this Issue or Its PR on github?

4

IMO it was a mistake to upgrade to a version that was so old, and it’s worth upgrading again ASAP. I’m not going to go through the last year’s worth of changes looking for a possible explanation.

1 Like
5

Aside from the log entry, are you seeing any issues with your cluster ? Is tripped always zero ?

If you have an appropriate license, then you can open a support case? I would suggest to upgrade. There are 2 possibilities - A) it has been fixed in a more recent release and you’’ll get that fix by upgrading. Or B) it remains a bug, you continue to see it, so someone will need to find and fix it, and to get that fix you’ll need to upgrade anyways! If you want to speed up the process, then you will likely need to help a bit. If you do nothing, and nobody else comes up with a better idea, you should not expect anything to change. Are you OK with that?

Outside of upgrading, see if you can correlate the breaker messages with anything in your workload, e.g. from any slowlog entries. What trace logging did you enable? In broad terms, what’s your cluster doing, and how loaded is it both on average and at peak load times? Is there anything “custom” added, or is it a vanilla installation?