# APM on docker did not send logs trace to elastic cloud

**URL:** <https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620>\
**Category:** APM\
**Tags:** docker, java\
**Created:** [January 27, 2022, 6:37pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620 "2022-01-27T18:37:38Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![bekk777](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bekk777/32/101315_2.png) [@bekk777](https://discuss.elastic.co/u/bekk777)\
**Post date:** [January 27, 2022, 6:37pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/1 "2022-01-27T18:37:38Z")

</div>

Hello everyone,  
I don´t understand why my configuration APM agent and server working on docker. But on the elastic cloud does not show anything on the APM service. But in Discover metrics show.  
cloud version v 7.15.2.  
Can someone please explain what I should do first? I'm new to elastic and little understand everything.  
This is like my problem.  
[https://discuss.elastic.co/t/how-to-use-apm-agent-for-hybris-application/164921](https://discuss.elastic.co/t/how-to-use-apm-agent-for-hybris-application/164921)

Thanks, advanced

---

<div class="post-metadata">

**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\
**Post date:** [February 1, 2022, 8:22am UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/2 "2022-02-01T08:22:55Z")

</div>

Hi @bekk777 , welcome to the forum.

The first thing to check is your agent setup, can you describe what you have done to add the agent so we can help you on the missing parts ?

For example, do you use the `-javaagent` JVM option, or rely on runtime attachment (either through API or through CLI ?). Do you see anything in your application logs and/or standard output ?

---

<div class="post-metadata">

**Author:** ![bekk777](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bekk777/32/101315_2.png) [@bekk777](https://discuss.elastic.co/u/bekk777)\
**Post date:** [February 1, 2022, 9:44am UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/3 "2022-02-01T09:44:33Z")

</div>

Hi Sylvain,  
Thank you answer. Yes sure  
This is my APM-Server which is running on docker.

```auto
apm-server:
        host: "0.0.0.0:8200"
output.elasticsearch:
        hosts: ["ip-elasticsearch:9200"]
        username: "elastic"
        password: "password"

```

Java agent configures like this manual: [Programmatic API setup to self-attach | APM Java Agent Reference [1.x] | Elastic](https://www.elastic.co/guide/en/apm/agent/java/current/setup-attach-api.html)  
Both of them are in docker run (APM-Server and java agent on the same network)  
Logs from APM-server:

```auto
2022-02-01T09:30:04.762Z DEBUG [elasticsearch] elasticsearch/client.go:719 Ping request failed with: Get http://ip-elasticsearch:9200: dial tcp ip-elasticsearch:9200: connect: connection refused
2022-02-01T09:30:20.649Z INFO [request] beater/common_handlers.go:272 handled request {"request_id": "85021106-a8cd-4d4c-9d4e-012bfecef9cf", "method": "POST", "URL": "/intake/v2/events", "content_length": -1, "remote_address": "docker-ip", "user-agent": "apm-agent-java/1.28.4 (test-agent-be)", "response_code": 202}
2022-02-01T09:30:20.650Z DEBUG [publish] pipeline/processor.go:309 Publish event: {
  "@timestamp": "2022-02-01T09:30:10.654Z",
  "@metadata": {
    "beat": "apm-server",
    "type": "doc",
    "version": "6.8.11"

```

I am something wrong, I am pretty sure. But 4 days I am working do everything I can 🙂  
The last hope is for you.  
Thank advance

---

<div class="post-metadata">

**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\
**Post date:** [February 1, 2022, 10:26am UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/4 "2022-02-01T10:26:27Z")

</div>

I'm not familiar with your Docker setup, but it looks like a network issue, are you using Docker compose or similar ?

`ip-elasticsearch` should be the DNS/IP name of the ES server and should be accessible from APM server container.

The apm-server container should be accessible from the application running with the agent. If your application needs to access a database backend the configuration should be similar here.

---

<div class="post-metadata">

**Author:** ![bekk777](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bekk777/32/101315_2.png) [@bekk777](https://discuss.elastic.co/u/bekk777)\
**Post date:** [February 1, 2022, 11:07am UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/5 "2022-02-01T11:07:52Z")

</div>

Thank you for replaying.  
I will check that, and send here feedback.

---

<div class="post-metadata">

**Author:** ![bekk777](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bekk777/32/101315_2.png) [@bekk777](https://discuss.elastic.co/u/bekk777)\
**Post date:** [February 1, 2022, 1:09pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/6 "2022-02-01T13:09:28Z")

</div>

On docker:

```auto
docker run --name=apm-server --network apm --dit docker.elastic.co/apm/apm-server:7.15.2 -p 8200:8200

```

And java agent was configured to listen apm-server:8200.

---

<div class="post-metadata">

**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\
**Post date:** [February 2, 2022, 9:04am UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/7 "2022-02-02T09:04:55Z")

</div>

The simplest solution to check for connection issues between the apm-agent and apm-server is to use the "health check" API endpoint on APM server with `curl`.

From the container that runs the application, you should use one of the following `curl` commands to check for connection issues. If this succeeds you should get an HTTP 200 response with JSON body.

For those examples, I'm assuming the name of apm-server is `apm-server` and that it's using plain `http`, not `https`.

```auto
# Without authentication
curl --request GET \
  --url http://apm-server:8200/

# When using secret token
curl --request GET \
  --url http://apm-server:8200/ \
  --header 'Authorization: Bearer <secret_token>'
 
# When using API key
curl --request GET \
  --url http://apm-server:8200/ \
  --header 'Authorization: ApiKey <api_key>'

```

---

<div class="post-metadata">

**Author:** ![bekk777](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bekk777/32/101315_2.png) [@bekk777](https://discuss.elastic.co/u/bekk777)\
**Post date:** [February 3, 2022, 2:11pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/8 "2022-02-03T14:11:32Z")

</div>

> [@Sylvain\_Juge](#):
>
> `curl`

Hi Sylvain.  
Thank you for your help.

When I checked on the app server (which is running on docker and java agent there too), I got this:

```auto
{"ok":{"build_date":"2020-07-09T18:01:36Z","build_sha":"dda44e48cd9c3bac7e5a359a14625284d1d10347","version":"6.8.11"}}root@ccd045e291db:

```

So, APM agent running correctly and request send from APM agent to APM server er correctly, right?  
But when I am sending from the APM server to the elastic server, I got this:

```auto
curl: (7) Failed connect to public-IP-elastic:8200; Connection refused

```

Request was: `curl public-IP-elastic:8200 -d '{"username": "elastic", "password": "secret_token"}' -v`

Something wrong between the APM server and elastic, right?

Regards,  
Nasimjon.

---

<div class="post-metadata">

**Author:** ![bekk777](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bekk777/32/101315_2.png) [@bekk777](https://discuss.elastic.co/u/bekk777)\
**Post date:** [February 3, 2022, 2:24pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/9 "2022-02-03T14:24:07Z")

</div>

In addition, metrics are coming to the elastic server. I saw in discovery part on Kibana. It is a surprise for me. 🙂

---

<div class="post-metadata">

**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\
**Post date:** [February 3, 2022, 3:23pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/10 "2022-02-03T15:23:58Z")

</div>

The URL/port of elastic server are not the same between apm agent and apm-server and between apm-server and elastic.

Can we conclude that it's now working as expected since you now have metrics ?

---

<div class="post-metadata">

**Author:** ![bekk777](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bekk777/32/101315_2.png) [@bekk777](https://discuss.elastic.co/u/bekk777)\
**Post date:** [February 3, 2022, 8:29pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/11 "2022-02-03T20:29:18Z")

</div>

> [@Sylvain\_Juge](#):
>
> Can we conclude that it's now working as expected since you now have metrics?

No 🙂 The metrics came right away when I was setting up the APM server :). But there is nothing in APM in services and transactions. What I wanna expecting. I am expecting this kind of service:  
[DEMO APM Services](https://demo.elastic.co/app/apm/services?rangeFrom=now-15m&rangeTo=now)

I believe that you are helping me.  
Thanks.

---

<div class="post-metadata">

**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\
**Post date:** [February 7, 2022, 8:06am UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/12 "2022-02-07T08:06:16Z")

</div>

The agent collects passively the application activity, unlike metrics which are collected through a scheduled task. I'm assuming that the agent setup works as expected since you are able to get metrics.

When you say "metrics are coming to the elastic server", can you describe in which index are they stored in ? Do you have a few sample documents ? There might be some system-metrics related metrics captured, and those are not the same as metrics captured through APM.

If you don't see transactions in Kibana, it means likely one of the following situations:

- the application has no activity, triggering a few transactions on the application is required
- the application relies on a technology that is not supported out of the box, checking [supported technologies](https://www.elastic.co/guide/en/apm/agent/java/current/supported-technologies-details.html)

---

<div class="post-metadata">

**Author:** ![bekk777](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bekk777/32/101315_2.png) [@bekk777](https://discuss.elastic.co/u/bekk777)\
**Post date:** [February 15, 2022, 9:12pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/13 "2022-02-15T21:12:45Z")

</div>

Hey Sylvain.  
Sorry, I didn't reply for a long time.  
This is a screenshot.

 ![Screenshot 2022-02-15 at 20.54.52](https://us1.discourse-cdn.com/elastic/original/3X/f/f/ffb412bd08d75b7d80632e2b50c1a4a30a8f59c1.png)

On APM-\* index is stored.

---

<div class="post-metadata">

**Author:** ![bekk777](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bekk777/32/101315_2.png) [@bekk777](https://discuss.elastic.co/u/bekk777)\
**Post date:** [February 20, 2022, 7:05pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/14 "2022-02-20T19:05:37Z")

</div>

Hey Sylvain,  
One question:  
Can I send APM-service not directly to ELK server, via Logstash, and maybe another solution?  
Thanks in advance.

---

<div class="post-metadata">

**Author:** ![Jack\_Shirazi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jack_shirazi/32/91641_2.png) [@Jack\_Shirazi](https://discuss.elastic.co/u/Jack_Shirazi)\
**Post date:** [February 22, 2022, 1:37pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/15 "2022-02-22T13:37:25Z")

</div>

Not sure what you are asking. The Java agent sends to an APM server (the URL you set) which then sends to Elasticsearch, you view the data that is held in Elasticsearch from kibana. The logs are not sent by the agent, and you need a different mechanism, like logstash, to have those sent

---

<div class="post-metadata">

**Author:** ![bekk777](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bekk777/32/101315_2.png) [@bekk777](https://discuss.elastic.co/u/bekk777)\
**Post date:** [February 22, 2022, 2:30pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/16 "2022-02-22T14:30:23Z")

</div>

Hey Jack,  
Thanks for replying.  
Yes, because when I send it to ELK-server, not show me anything on services APM, transaction. I think maybe it will be possible for the first to handle it (to Logstash from APM-server) and then send it to the ELK server.  
This is a diagram of my APM server.

 ![ELK](https://us1.discourse-cdn.com/elastic/original/3X/9/b/9bec37a22dad441ef181ae3cc2e4fe63746a96e1.jpeg)

How can I configure like DEMO link, which is mentioned above? ⬆

---

<div class="post-metadata">

**Author:** ![Jack\_Shirazi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jack_shirazi/32/91641_2.png) [@Jack\_Shirazi](https://discuss.elastic.co/u/Jack_Shirazi)\
**Post date:** [February 23, 2022, 1:56pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/17 "2022-02-23T13:56:31Z")

</div>

Can we clarify exactly what we have and what is missing.

1. You have a Java running application with a Java agent which is self-attaching (I assume you modified your main class to do this). I assume that the Java application is showing correct functionality. The Java application is running in a docker container
2. You have an APM agent running in a (different? the same?) docker container on the default 8200 port. The Java agent is configured to point to the APM agent on localhost 8200.
3. You can curl successfully (from inside the docker container?) to the APM agent (curl [http://localhost:8200](http://localhost:8200) ?) and the APM agent reports it is running version 6.8.11 (which is a very old version)
4. There are metrics being received by the Elasticsearch node, including one example metric which includes garbage collector name and heap stats (I assume this is from the Java application, ie that there is no other Java application running against the system with the given hostname). But this is a beat metric, not a Java agent metric.
5. You are expecting to see a transaction but you are not seeing that  
Please correct anything that is incorrect.

Have you turned [debug log level](https://www.elastic.co/guide/en/apm/agent/java/current/config-logging.html#config-log-level) on for the agent to see if the Java agent is connecting and instrumenting what you expect?

Exactly what transaction are you expecting to see? If your Java application is some custom classes which execute custom functionality, the Java agent does not know what to call a transaction. The out-of-the-box technologies the agent will instrument are [Supported technologies | APM Java Agent Reference [1.x] | Elastic](https://www.elastic.co/guide/en/apm/agent/java/current/supported-technologies-details.html) - otherwise the agent needs you to explicitly specify the transactions

---

<div class="post-metadata">

**Author:** ![bekk777](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bekk777/32/101315_2.png) [@bekk777](https://discuss.elastic.co/u/bekk777)\
**Post date:** [February 23, 2022, 3:07pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/18 "2022-02-23T15:07:07Z")

</div>

Hey Jack.  
Thanks for your help.  
Yes, you are absolutely correct. But one thing to be more clear: Docker is running two containers that have a single network (same network). The Java agent is configured to listen on the network with the APM-server port, for example, in my case, apm-server:8200. And the APM server sends further to the ElK-server.  
I am not sure about "debug log level". Today I'll take a look and give you an answer to this.  
In addition, if I do all these procedures from my PC, I send it to the elk server, everything works for me (Example). What I want. But with docker, no.

---

<div class="post-metadata">

**Author:** ![bekk777](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bekk777/32/101315_2.png) [@bekk777](https://discuss.elastic.co/u/bekk777)\
**Post date:** [February 24, 2022, 4:48pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/19 "2022-02-24T16:48:40Z")

</div>

From java agent i got this: `ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Failed to handle event of type JSON_WRITER with this error: Connection refused`  
From apm server i got this: `2022-02-24T16:38:07.428Z	INFO	pipeline/output.go:105	Connection to backoff(elasticsearch(http://elasticsearch-host:9200)) established 2022-02-24T16:38:37.766Z	INFO	[request]	beater/common_handlers.go:272	handled request	{"request_id": "ca9cc236-7527-4899-8bd5-c7237ddf0f2c", "method": "POST", "URL": "/intake/v2/events", "content_length": -1, "remote_address": "172.19.0.2", "user-agent": "apm-agent-java/1.28.4 (java-app)", "response_code": 202}`

---

<div class="post-metadata">

**Author:** ![Jack\_Shirazi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jack_shirazi/32/91641_2.png) [@Jack\_Shirazi](https://discuss.elastic.co/u/Jack_Shirazi)\
**Post date:** [February 24, 2022, 5:37pm UTC](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620/20 "2022-02-24T17:37:32Z")

</div>

So that says the Java agent is NOT connecting to the APM agent in the same docker - you have some kind of inter-container connection problem, either the port is mapped wrong or the network is not the same. This is consistent with the rest of what you see - the APM agent has connected to Elasticsearch and has sent data (that beat metric), but the Java agent has not connected to the APM agent so you see nothing from it.  
I can't help further here, this is docker specific and depends on how you have setup your containers

[Next page](https://discuss.elastic.co/t/apm-on-docker-did-not-send-logs-trace-to-elastic-cloud/295620.md?page=2)