# Instrumentation of Jenkins fails, a thread dies unexpectedly due to an uncaught exception

**URL:** <https://discuss.elastic.co/t/instrumentation-of-jenkins-fails-a-thread-dies-unexpectedly-due-to-an-uncaught-exception/189097>\
**Category:** APM\
**Tags:** java\
**Created:** [July 5, 2019, 12:37pm UTC](https://discuss.elastic.co/t/instrumentation-of-jenkins-fails-a-thread-dies-unexpectedly-due-to-an-uncaught-exception/189097 "2019-07-05T12:37:09Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![leprovokateur](https://avatars.discourse-cdn.com/v4/letter/l/8e8cbc/32.png) [@leprovokateur](https://discuss.elastic.co/u/leprovokateur)\
**Post date:** [July 5, 2019, 12:37pm UTC](https://discuss.elastic.co/t/instrumentation-of-jenkins-fails-a-thread-dies-unexpectedly-due-to-an-uncaught-exception/189097/1 "2019-07-05T12:37:09Z")

</div>

Hi,

I tried to instrument a jenkins instance, but the thread for the elastic apm agent dies.

**Kibana version** : 7.2

**Elasticsearch version** : 7.2

**APM Server version** : 7.2

**APM Agent language and version** : Java, 1.7

**Steps to reproduce** :

1. Install Jenkins
2. Add the following to JENKINS\_JAVA\_OPTIONS:  
-javaagent:/opt/elastic-apm-agent-1.7.0.jar -Delastic.apm.disable\_instrumentation='' -Delastic.apm.application\_packages=hudson,jenkins,org.eclipse -Delastic.apm.trace\_methods=hudson._,jenkins._,org.eclipse.\* -Delastic.apm.service\_name=jenkins -Delastic.apm.server\_url=http://apm-server:8200"
3. Restart Jenkins

**Provide logs and/or server output (if relevant)**:  
2019-07-05 12:25:55.744+0000 [id=13] SEVERE h.i.i.InstallUncaughtExceptionHandler$DefaultUncaughtExceptionHandler#uncaughtException: A thread (apm-request-timeout-timer/13) died unexpectedly due to an uncaught exception, this may leave your Jenkins in a bad way and is usually indicative of a bug in the code.  
java.lang.IllegalStateException: Ring buffer has no available slots  
at co.elastic.apm.agent.report.ApmServerReporter.flush(ApmServerReporter.java:173)  
at co.elastic.apm.agent.report.IntakeV2ReportingEventHandler$FlushOnTimeoutTimerTask.run(IntakeV2ReportingEventHandler.java:412)  
at java.base/java.util.TimerThread.mainLoop(Timer.java:556)  
at java.base/java.util.TimerThread.run(Timer.java:506)  
2019-07-05 14:26:21.329 [apm-reporter] INFO co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Backing off for 0 seconds (+/-10%)  
2019-07-05 14:26:21.329 [apm-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Error sending data to APM server: Error writing request body to server, response code is -1  
2019-07-05 14:26:21.330 [apm-reporter] WARN co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - null  
2019-07-05 14:26:21.332 [apm-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Failed to handle event of type SPAN with this error: Timer already cancelled.  
2019-07-05 14:26:21.333 [apm-reporter] INFO co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Backing off for 1 seconds (+/-10%)

Any ideas?

Best regards,  
Robert

---

<div class="post-metadata">

**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\
**Post date:** [July 7, 2019, 3:44am UTC](https://discuss.elastic.co/t/instrumentation-of-jenkins-fails-a-thread-dies-unexpectedly-due-to-an-uncaught-exception/189097/2 "2019-07-07T03:44:10Z")

</div>

Hi and thanks for reporting!

Looks like you are right- the Timer behaves as if it was cancelled when the main loop throws an Exception. We will look into that.

Does this reproduces every time?  
Was there a proper connection with the APM server prior to that? You can see that in the top of the agent log.

---

<div class="post-metadata">

**Author:** ![leprovokateur](https://avatars.discourse-cdn.com/v4/letter/l/8e8cbc/32.png) [@leprovokateur](https://discuss.elastic.co/u/leprovokateur)\
**Post date:** [July 10, 2019, 1:39pm UTC](https://discuss.elastic.co/t/instrumentation-of-jenkins-fails-a-thread-dies-unexpectedly-due-to-an-uncaught-exception/189097/3 "2019-07-10T13:39:07Z")

</div>

Hi,

This happened once shortly after a restart of the jenkins, today it run fine for several hours, but crashed now.

Shortly before the crash, lines like the following appeared in the log:

2019-07-10 15:10:30.924 [apm-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Error sending data to APM server: Server returned HTTP response code: 503 for URL: [http://192.168.122.150:8200/intake/v2/events](http://192.168.122.150:8200/intake/v2/events), response code is 503  
2019-07-10 15:10:30.924 [apm-reporter] WARN co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - {  
"accepted": 890,  
"errors": [  
{  
"message": "queue is full"  
}  
]  
}

---

<div class="post-metadata">

**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\
**Post date:** [July 10, 2019, 2:05pm UTC](https://discuss.elastic.co/t/instrumentation-of-jenkins-fails-a-thread-dies-unexpectedly-due-to-an-uncaught-exception/189097/4 "2019-07-10T14:05:02Z")

</div>

Oh no!  
Please send the log from this error message and until the end, and any other server log, if there's such, that may contain info about the crash.

For what i saw before I made [a pull request](https://github.com/elastic/apm-agent-java/pull/704). Try using it's product - [this snapshot build](https://apm-ci.elastic.co/job/apm-agent-java/job/apm-agent-java-mbp/job/PR-704/2/artifact/src/github.com/elastic/apm-agent-java/elastic-apm-agent/target/elastic-apm-agent-1.7.1-SNAPSHOT.jar).

---

<div class="post-metadata">

**Author:** ![leprovokateur](https://avatars.discourse-cdn.com/v4/letter/l/8e8cbc/32.png) [@leprovokateur](https://discuss.elastic.co/u/leprovokateur)\
**Post date:** [July 10, 2019, 2:16pm UTC](https://discuss.elastic.co/t/instrumentation-of-jenkins-fails-a-thread-dies-unexpectedly-due-to-an-uncaught-exception/189097/5 "2019-07-10T14:16:34Z")

</div>

I'll have the new version in use and report back later.

---

<div class="post-metadata">

**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\
**Post date:** [July 10, 2019, 2:44pm UTC](https://discuss.elastic.co/t/instrumentation-of-jenkins-fails-a-thread-dies-unexpectedly-due-to-an-uncaught-exception/189097/6 "2019-07-10T14:44:54Z")

</div>

Thanks, though note it is not an official version, it's a snapshot build

---

<div class="post-metadata">

**Author:** ![leprovokateur](https://avatars.discourse-cdn.com/v4/letter/l/8e8cbc/32.png) [@leprovokateur](https://discuss.elastic.co/u/leprovokateur)\
**Post date:** [July 12, 2019, 6:52am UTC](https://discuss.elastic.co/t/instrumentation-of-jenkins-fails-a-thread-dies-unexpectedly-due-to-an-uncaught-exception/189097/7 "2019-07-12T06:52:25Z")

</div>

So, it seems, that has fixed the issue.

---

<div class="post-metadata">

**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\
**Post date:** [July 14, 2019, 3:53am UTC](https://discuss.elastic.co/t/instrumentation-of-jenkins-fails-a-thread-dies-unexpectedly-due-to-an-uncaught-exception/189097/8 "2019-07-14T03:53:30Z")

</div>

Great! Thanks for the update.  
Please let us know if something changes.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 3, 2019, 11:55pm UTC](https://discuss.elastic.co/t/instrumentation-of-jenkins-fails-a-thread-dies-unexpectedly-due-to-an-uncaught-exception/189097/9 "2019-08-03T23:55:51Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.
