# Standalone apm-server with elasticsearch 8 in docker not able to connect

**URL:** <https://discuss.elastic.co/t/standalone-apm-server-with-elasticsearch-8-in-docker-not-able-to-connect/319892>\
**Category:** APM\
**Tags:** docker, server\
**Created:** [November 27, 2022, 8:56pm UTC](https://discuss.elastic.co/t/standalone-apm-server-with-elasticsearch-8-in-docker-not-able-to-connect/319892 "2022-11-27T20:56:19Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![AlanMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alanmark/32/56292_2.png) [@AlanMark](https://discuss.elastic.co/u/AlanMark)\
**Post date:** [November 27, 2022, 8:56pm UTC](https://discuss.elastic.co/t/standalone-apm-server-with-elasticsearch-8-in-docker-not-able-to-connect/319892/1 "2022-11-27T20:56:19Z")

</div>

**Elasticsearch version** : 8.5.0

**APM Server version** : 8.5.2

**Original install method (e.g. download page, yum, deb, from source, etc.) and version**: docker

**Fresh install or upgraded from other version?** : fresh

**Setup** :  
Attempting to run apm-server standalone, ie. not using an elastic agent, and on docker. This worked fine in 7.x, but in 8.x I'm having all sorts of problems.

Using the following setup:

```auto
docker run -d \
  -p 8200:8200 \
  --network=elastic \
  --name=apm-server \
  --user=apm-server \
  --volume="$(pwd)/ca.crt:/usr/share/apm-server/ca.crt:ro" \
  docker.elastic.co/apm/apm-server:8.5.2 \
  --strict.perms=false -e \
  -E output.elasticsearch.hosts=["https://es00:9200"] \
  -E output.elasticsearch.api_key="some_api_key_for_elastic_user" \
  -E output.elasticsearch.ssl.certificate_authorities="/usr/share/apm-server/ca.crt"

```

Elasticsearch is running on the same cluster in the same network. I am using the same CA. The API key is generated with the elastic master user.

**Problem** :  
APM is unable to connect, and it gives a very non-descript error that looks faulty (notice the missing uuid):

```auto
apm-server | {"log.level":"error","@timestamp":"2022-11-27T20:26:57.297Z","log.logger":"beater","log.origin":{"file.name":"beater/waitready.go","file.line":64},"message":"precondition failed: error querying cluster_uuid: status_code=401","service.name":"apm-server","ecs.version":"1.6.0"}

```

I have verified that I can reach both containers from eachothers hostnames on the docker network.

I also get the exact same error if i use username / password approach with the elastic user.

I'm aware that the elastic user is bad practice here - I'm just working in a test environment and need to verify settings before hardening.

The remaining log looks like this:

```auto
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.287Z","log.origin":{"file.name":"instance/beat.go","file.line":708},"message":"Home path: [/usr/share/apm-server] Config path: [/usr/share/apm-server] Data path: [/usr/share/apm-server/data] Logs path: [/usr/share/apm-server/logs]","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"debug","@timestamp":"2022-11-27T20:26:32.287Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":766},"message":"Beat metadata path: /usr/share/apm-server/data/meta.json","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.287Z","log.origin":{"file.name":"instance/beat.go","file.line":716},"message":"Beat ID: be37df93-d182-4a1a-822a-777e4f97548b","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.288Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1082},"message":"Beat info","service.name":"apm-server","system_info":{"beat":{"path":{"config":"/usr/share/apm-server","data":"/usr/share/apm-server/data","home":"/usr/share/apm-server","logs":"/usr/share/apm-server/logs"},"type":"apm-server","uuid":"be37df93-d182-4a1a-822a-777e4f97548b"},"ecs.version":"1.6.0"}}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.288Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1091},"message":"Build info","service.name":"apm-server","system_info":{"build":{"commit":"603b5878ad5a02225c03970741b87fb7f8c38e4a","libbeat":"8.5.2","time":"2022-11-17T10:57:37.000+0100","version":"8.5.2"},"ecs.version":"1.6.0"}}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.288Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1094},"message":"Go runtime info","service.name":"apm-server","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":32,"version":"go1.18.5"},"ecs.version":"1.6.0"}}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.288Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1098},"message":"Host info","service.name":"apm-server","system_info":{SCRUBBED}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.288Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1127},"message":"Process info","service.name":"apm-server","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":null,"effective":null,"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null},"cwd":"/usr/share/apm-server","exe":"/usr/share/apm-server/apm-server","name":"apm-server","pid":7,"ppid":1,"seccomp":{"mode":"filter","no_new_privs":false},"start_time":"2022-11-27T20:26:31.910Z"},"ecs.version":"1.6.0"}}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.288Z","log.origin":{"file.name":"instance/beat.go","file.line":294},"message":"Setup Beat: apm-server; Version: 8.5.2","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"debug","@timestamp":"2022-11-27T20:26:32.289Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":319},"message":"Initializing output plugins","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"warn","@timestamp":"2022-11-27T20:26:32.290Z","log.logger":"cfgwarn","log.origin":{"file.name":"tlscommon/config.go","file.line":102},"message":"DEPRECATED: Treating the CommonName field on X.509 certificates as a host name when no Subject Alternative Names are present is going to be removed. Please update your certificates if needed. Will be removed in version: 8.0.0","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"debug","@timestamp":"2022-11-27T20:26:32.290Z","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls.go","file.line":172},"message":"Successfully loaded CA certificate: /usr/share/apm-server/ca.crt","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.290Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":108},"message":"elasticsearch url: https://es00:9200","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"debug","@timestamp":"2022-11-27T20:26:32.291Z","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls.go","file.line":172},"message":"Successfully loaded CA certificate: /usr/share/apm-server/ca.crt","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"debug","@timestamp":"2022-11-27T20:26:32.291Z","log.logger":"publisher","log.origin":{"file.name":"pipeline/consumer.go","file.line":98},"message":"start pipeline event consumer","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.291Z","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: 825bd2425dff","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"debug","@timestamp":"2022-11-27T20:26:32.291Z","log.logger":"publisher","log.origin":{"file.name":"pipeline/queue_reader.go","file.line":49},"message":"pipeline event consumer queue reader: start","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.291Z","log.origin":{"file.name":"instance/beat.go","file.line":471},"message":"apm-server start running.","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.292Z","log.logger":"beater","log.origin":{"file.name":"beater/http.go","file.line":142},"message":"Listening on: [::]:8200","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.292Z","log.logger":"beater","log.origin":{"file.name":"beater/beater.go","file.line":1162},"message":"maxprocs: Leaving GOMAXPROCS=32: CPU quota undefined","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"debug","@timestamp":"2022-11-27T20:26:32.292Z","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls.go","file.line":172},"message":"Successfully loaded CA certificate: /usr/share/apm-server/ca.crt","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"debug","@timestamp":"2022-11-27T20:26:32.292Z","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls.go","file.line":172},"message":"Successfully loaded CA certificate: /usr/share/apm-server/ca.crt","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.292Z","log.logger":"beater","log.origin":{"file.name":"beater/waitready.go","file.line":40},"message":"blocking ingestion until all preconditions are satisfied","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.293Z","log.logger":"beater","log.origin":{"file.name":"apm-server/main.go","file.line":71},"message":"creating transaction metrics aggregation with config: {Interval:1m0s MaxTransactionGroups:10000 HDRHistogramSignificantFigures:2}","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.296Z","log.logger":"beater","log.origin":{"file.name":"apm-server/main.go","file.line":86},"message":"creating service destinations aggregation with config: {Interval:1m0s MaxGroups:10000}","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.296Z","log.logger":"handler","log.origin":{"file.name":"api/mux.go","file.line":133},"message":"Path / added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.296Z","log.logger":"handler","log.origin":{"file.name":"api/mux.go","file.line":133},"message":"Path /config/v1/agents added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.296Z","log.logger":"handler","log.origin":{"file.name":"api/mux.go","file.line":133},"message":"Path /config/v1/rum/agents added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.296Z","log.logger":"handler","log.origin":{"file.name":"api/mux.go","file.line":133},"message":"Path /intake/v2/rum/events added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.296Z","log.logger":"handler","log.origin":{"file.name":"api/mux.go","file.line":133},"message":"Path /intake/v3/rum/events added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.296Z","log.logger":"handler","log.origin":{"file.name":"api/mux.go","file.line":133},"message":"Path /intake/v2/events added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.296Z","log.logger":"handler","log.origin":{"file.name":"api/mux.go","file.line":133},"message":"Path /v1/traces added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.296Z","log.logger":"handler","log.origin":{"file.name":"api/mux.go","file.line":133},"message":"Path /v1/metrics added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.296Z","log.logger":"handler","log.origin":{"file.name":"api/mux.go","file.line":133},"message":"Path /v1/logs added to request handler","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.296Z","log.logger":"beater","log.origin":{"file.name":"beater/server.go","file.line":193},"message":"Starting apm-server [603b5878ad5a02225c03970741b87fb7f8c38e4a built 2022-11-17 10:57:37 +0100 +0100]. Hit CTRL-C to stop it.","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.296Z","log.logger":"beater","log.origin":{"file.name":"beater/http.go","file.line":94},"message":"RUM endpoints disabled.","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"info","@timestamp":"2022-11-27T20:26:32.296Z","log.logger":"beater","log.origin":{"file.name":"beater/http.go","file.line":104},"message":"SSL disabled.","service.name":"apm-server","ecs.version":"1.6.0"}
apm-server | {"log.level":"error","@timestamp":"2022-11-27T20:26:32.297Z","log.logger":"beater","log.origin":{"file.name":"beater/waitready.go","file.line":64},"message":"precondition failed: error querying cluster_uuid: status_code=401","service.name":"apm-server","ecs.version":"1.6.0"}

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 28, 2022, 12:10am UTC](https://discuss.elastic.co/t/standalone-apm-server-with-elasticsearch-8-in-docker-not-able-to-connect/319892/2 "2022-11-28T00:10:17Z")

</div>

Perhaps look at the compose file [here](https://www.elastic.co/guide/en/apm/guide/current/quick-start-overview.html)

Is this a brand new cluster?... I think you might need that the setup config etc

Having trouble replicating right now.. haven't had a chance to set up.

Changed: Question did you actually add the APM Integration you need to do that to / add the assets even if you are not using fleet .. that is where the templates get loaded etc.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 28, 2022, 1:45am UTC](https://discuss.elastic.co/t/standalone-apm-server-with-elasticsearch-8-in-docker-not-able-to-connect/319892/3 "2022-11-28T01:45:18Z")

</div>

Ok I got mine working...

I think you did not setup / load the APM integration... kibana weird but you still need to do that... you need to do that **before** you start the APM Server

 ![Screen Shot 2022-11-27 at 5.38.09 PM](https://us1.discourse-cdn.com/elastic/original/3X/9/7/9772861a5d0e176f3c311532dbcdcbba6246b159.jpeg)

Then I see the assets

```auto
GET _cat/indices/.*?v
GET _cat/templates/tra*

# GET _cat/indices/.*?v 200 OK
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .fleet-enrollment-api-keys-7 0JXPxyXpRFarcWU-7QBuBw 1 0 1 0 6.1kb 6.1kb
green open .apm-agent-configuration Y2n3cINATZyg9xhOVsJmmw 1 0 0 0 225b 225b
green open .kibana_security_session_1 dFOtc-OnTMqevxUGc9p5tw 1 0 1 0 5.7kb 5.7kb
green open .kibana-event-log-8.4.0-000001 LWUfHiPDShyzT_POBu23SA 1 0 1 0 6.2kb 6.2kb
green open .geoip_databases C_e4xhUBSQa5IrvzeBfUJQ 1 0 41 0 39.2mb 39.2mb
green open .security-7 SxKjtS3wQy2V4UU-G3q5CA 1 0 111 0 336kb 336kb
green open .kibana_task_manager_8.4.0_001 --xVYRnoSt2H4yKCNPD7vw 1 0 25 1397 238.1kb 238.1kb
green open .apm-custom-link 3lBvbBxQSgW6bd4PWKfiiQ 1 0 0 0 225b 225b
green open .kibana_8.4.0_001 jBOCGWxGQT-iTKg5OZ_QOA 1 0 2385 1226 8.9mb 8.9mb
green open .security-profile-8 NT5vs6sOQfW113PXPh0zvA 1 0 1 0 7.7kb 7.7kb
green open .fleet-policies-7 Sz0IgOoDSuepT3uvZBzEoQ 1 0 3 0 22kb 22kb
green open .ds-.logs-deprecation.elasticsearch-default-2022.11.28-000001 cQDV1sFnSyyrDGAOGImXNg 1 0 1 0 12.3kb 12.3kb
green open .ds-ilm-history-5-2022.11.28-000001 W5owRm08QAmWpKbotz9DrQ 1 0 9 0 35.6kb 35.6kb

# GET _cat/templates/tra* 200 OK
traces-apm [traces-apm-*] 200 [traces-apm@package, traces-apm@custom, .fleet_globals-1, .fleet_agent_id_verification-1]
traces-apm.sampled [traces-apm.sampled-*] 200 [traces-apm.sampled@package, traces-apm.sampled@custom, .fleet_globals-1, .fleet_agent_id_verification-1]
traces-apm.rum [traces-apm.rum-*] 200 [traces-apm.rum@package, traces-apm.rum@custom, .fleet_globals-1, .fleet_agent_id_verification-1]

```

Then my apm-server.docker.yml looks like this

```auto
apm-server:
  # Defines the host and port the server is listening on. Use "unix:/path/to.sock" to listen on a unix domain socket.
  host: "0.0.0.0:8200"

#-------------------------- Elasticsearch output --------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  # Scheme and port can be left out and will be set to the default (`http` and `9200`).
  # In case you specify and additional path, the scheme is required: `http://elasticsearch:9200/path`.
  # IPv6 addresses should always be defined as: `https://[2001:db8::1]:9200`.
  hosts: ["https://host.docker.internal:9200"]
  ssl.verification_mode: none
  username: "elastic"
  password: "password"

```

Then I ran...

```auto
docker run -d \
  -p 8200:8200 \
  --name=apm-server \
  --user=apm-server \
  --volume="$(pwd)/apm-server.docker.yml:/usr/share/apm-server/apm-server.yml:ro" \
  docker.elastic.co/apm/apm-server:8.5.2 \
  --strict.perms=false -e

```

Then I got this

 ![Screen Shot 2022-11-27 at 5.42.49 PM](https://us1.discourse-cdn.com/elastic/original/3X/3/3/332536e07ed62732311b135a2b1fe3d3a0341167.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 18, 2022, 9:46pm UTC](https://discuss.elastic.co/t/standalone-apm-server-with-elasticsearch-8-in-docker-not-able-to-connect/319892/4 "2022-12-18T21:46:17Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.