# Java service not recovering from APM circuit breaker after CPU stress subsides in kubernetes

**URL:** <https://discuss.elastic.co/t/java-service-not-recovering-from-apm-circuit-breaker-after-cpu-stress-subsides-in-kubernetes/352082>\
**Category:** APM\
**Tags:** java\
**Created:** [January 30, 2024, 12:00pm UTC](https://discuss.elastic.co/t/java-service-not-recovering-from-apm-circuit-breaker-after-cpu-stress-subsides-in-kubernetes/352082 "2024-01-30T12:00:42Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![HemdeepSaini](https://avatars.discourse-cdn.com/v4/letter/h/13edae/32.png) [@HemdeepSaini](https://discuss.elastic.co/u/HemdeepSaini)\
**Post date:** [January 30, 2024, 12:00pm UTC](https://discuss.elastic.co/t/java-service-not-recovering-from-apm-circuit-breaker-after-cpu-stress-subsides-in-kubernetes/352082/1 "2024-01-30T12:00:43Z")

</div>

**Description:**

I'm using Elastic APM to monitor a Java application deployed on a Kubernetes pod. When the service initially experiences CPU stress, the APM `TRACER` correctly switches to the `PAUSED` state due to the circuit breaker. However, even after the CPU load falls below the configured threshold, the `TRACER` remains in the `PAUSED` state, and traces and metrics are not captured.

**Expected Behavior:**

The APM tracer should automatically switch back to the `RUNNING` state once the CPU load returns to normal levels. This would allow APM to resume capturing traces for the service.

**Current Behavior:**

- The CPU load initially spikes and triggers the circuit breaker, pausing the tracer.
- The CPU load subsequently returns to normal levels, but the tracer remains paused.
- Traces are not captured while the tracer is paused.

**Logs:**

```auto
 2024-01-30 14:37:00,511 [elastic-apm-circuit-breaker] INFO co.elastic.apm.agent.impl.circuitbreaker.CircuitBreaker - Stress detected by co.elastic.apm.agent.impl.circuitbreaker.SystemCpuStressMonitor: Latest system CPU load value measured is 1.0. This is the 20th consecutive measurement that crossed the configured stress threshold - 0.95, which indicates this host is under CPU stress. 2024-01-30 14:37:00,518 [elastic-apm-circuit-breaker] INFO co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to PAUSED state

```

**Environment:**

- Kibana version: 8.8.1
- Elasticsearch version: 8.8.0
- APM Server version: 8.8.3
- APM Agent language and version: (Please specify language and version)

**Additional Information:**

 ![Screenshot from 2024-01-30 17-09-00](https://us1.discourse-cdn.com/elastic/original/3X/8/5/85f9c4a47ef1650bd522ea8e0becf1948d7ffeb0.png)  
as seen in picture cpu stress subsides

- stress\_monitor\_gc\_relief\_threshold = 0.9
- stress\_monitor\_gc\_stress\_threshold = 0.95

Why is the APM tracer not automatically switching back to the `RUNNING` state in k8 after the CPU stress subsides? Is there any additional configuration or troubleshooting steps I can take to address this issue?

---

<div class="post-metadata">

**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\
**Post date:** [January 30, 2024, 1:01pm UTC](https://discuss.elastic.co/t/java-service-not-recovering-from-apm-circuit-breaker-after-cpu-stress-subsides-in-kubernetes/352082/2 "2024-01-30T13:01:09Z")

</div>

Hi @HemdeepSaini ,

The circuit breaker is expected to return to a normal state after the load spike has passed.

In order to investigate this you should increase the agent log level with `log_level=debug` or even `log_level=trace` to see what is happening.

The related log messages you will see in the logs will be from the [CircuitBreaker class](https://github.com/elastic/apm-agent-java/blob/864e4e2cbdd0177c6fcbd7eca38a2f0ba8167463/apm-agent-core/src/main/java/co/elastic/apm/agent/impl/circuitbreaker/CircuitBreaker.java#L104)

Also, the agent uses polling, thus there is a delay before it should detect that the load is now lower.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 27, 2024, 1:01pm UTC](https://discuss.elastic.co/t/java-service-not-recovering-from-apm-circuit-breaker-after-cpu-stress-subsides-in-kubernetes/352082/3 "2024-02-27T13:01:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.