We are using the APM Java agent and attaching it programatically using "ElasticApmAttacher.attach()".
It is running more then a year now but the last months we are experiencing problems.
Java Agent is not able to send data to Elasticsearch. The following exception is found thousands of times in the weblogic log:
After restart of the weblogic server the proces continues but very soon (sometimes a couple of hours sometimes a couple of days later) the same issue appears with the same error log.
I could not find any clues searching the internet.
Any ideas? Where to start to analyse this problem?
I don't have a stack trace yet. Not showing up in the weblogic logs.
But I have requested the Elasticsearch logs from the operators. This could take a couple of days unfortunately.
Should I also have the Kibana logs?
It started to happen more frequently the last 3 months. Last time, this happened within hours after restarting weblogic.
A couple of months ago it worked for some weeks after a restart.
When it happens it is constantly throwing this error thousands of times per day.
The Elasticsearch and Kibana logs are not relevant here, what you need to provide is the elastic apm agent logs.
A common procedure to capture those in a separate file is available here.
Given you are deploying on Weblogic, does it means one webapp deployed on the server deploys the agent through runtime attach with the API, and then it instruments the whole service ?
As an alternative, trying to use the more simple -javaagent setup method could likely help you achieve better stability. While we haven't changed this part recently it might still have a few corner cases.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.