Jav APM Agent Self Attach cli jar is not working inside the container

Elastic Observability APM
, ,
1

Hello,

My application is running inside a docker container. If i use jvm args for apm agent attached like below, working fine

javaagent:/opt/elastic-apm-agent.jar -Delastic.apm.service_name=myApp -Delastic.apm.server_url=http://test:8200

i am trying to use self-attach as per the below documentation, which mentioned Use this script to automatically attach to all docker containers running on a host. not able to get it up.

What i tried:

Created a selfattach.sh file inside root folder of the container with below:

#!/usr/bin/env bash
set -ex

attach () {
    # only attempt attachment if this looks like a java container
    if [[ $(docker inspect ${container_id} | jq --raw-output .[0].Config.Cmd[0]) == java ]]
    then
        echo attaching to $(docker ps --no-trunc | grep ${container_id})
        docker cp ./apm-agent-attach-*-cli.jar ${container_id}:/apm-agent-attach-cli.jar
        docker exec ${container_id} java -jar /apm-agent-attach-cli.jar --config service_name=App_Perf --config server_url=http://Test:8200
    fi
}

# attach to running containers
for container_id in $(docker ps --quiet --no-trunc) ; do
    attach
done

# listen for starting containers and attach to those
docker events --filter 'event=start' --format '{{.ID}}' |
while IFS= read -r container_id
do
    attach
done

Tried to run the bash file, getting below error, not sure what this related to. Request to let me know the fix.

Caused by: java.nio.file.AccessDeniedException: /tmp/whoami5350918110509627295.tmp

The container is running as eoadmin user and he has access to the root folder of the container.

2

Hi @Bhanu_Praveen ,

If I understand this correctly, you are trying to use the CLI attacher to automatically attach to JVMs that run within docker containers. This shell script is provided as an example only, and there aren't any strong guarantees it works in all cases. As mentioned in this page, this script is experimental.

For attach to work properly, the attach process needs to have the ability to run with exactly the same user and permissions as the user running the process, which is something that can be tricky to achieve with containers.

As a simpler alternative, I would suggest to change the container environment variables and set the JAVA_TOOL_OPTIONS to include the -javaagent: argument, which can be done without modifying the container.

3

Ya, that's what we are doing for individual jvms by setting below in jvm args. Looking to self attach instead of restarting my jvm and on-demand enabled/disable APM metrics pull.

-javaagent:/opt/trade/elastic-apm-agent.jar -Delastic.apm.service_name=GTMApp -Delastic.apm.server_url=http://test:8200

4

Maybe as an alternative, you could always add the agent, but use the recording configuration option.

When set to true the agent will record and send data, when set to false it won't send anything.

The advantage here is that you can change dynamically this setting at runtime without restart using either central configuration (remotely) or when agent configuration is in a properties file that you can edit at runtime.

When the agent is attached there is always some overhead involved due to instrumentation, so the application response time tends to be impacted when this happens, if it's already under load it won't help.