Error while launching APM server

Elastic Observability APM
1

Hi,
I have installed and launched Elasticsearch version 7.5.2 and kibana 7.5.2 on same server and I am trying to run apm server 7.5.2 but it is giving below error while running ./apm-server -e command.

I am using chrome browser and installation was done by downloading the packages and installing offline.

PFB the logs of APM server startup :

./apm-server -e
2020-02-11T12:08:14.249+0530 INFO instance/beat.go:610 Home path: [/utxciserver/Elastic/elastic_7.5.2/apm-server-7.5.2-linux-x86] Config path: [/utxciserver/Elastic/elastic_7.5.2/apm-server-7.5.2-linux-x86] Data path: [/utxciserver/Elastic/elastic_7.5.2/apm-server-7.5.2-linux-x86/data] Logs path: [/utxciserver/Elastic/elastic_7.5.2/apm-server-7.5.2-linux-x86/logs]
2020-02-11T12:08:14.271+0530 INFO instance/beat.go:618 Beat ID: 1f9397f6-2eec-4c16-81f0-748a69f472fb
2020-02-11T12:08:14.283+0530 INFO [seccomp] seccomp/seccomp.go:101 Syscall filter could not be installed because the kernel does not support seccomp
2020-02-11T12:08:14.290+0530 INFO [beat] instance/beat.go:941 Beat info {"system_info": {"beat": {"path": {"config": "/utxciserver/Elastic/elastic_7.5.2/apm-server-7.5.2-linux-x86", "data": "/utxciserver/Elastic/elastic_7.5.2/apm-server-7.5.2-linux-x86/data", "home": "/utxciserver/Elastic/elastic_7.5.2/apm-server-7.5.2-linux-x86", "logs": "/utxciserver/Elastic/elastic_7.5.2/apm-server-7.5.2-linux-x86/logs"}, "type": "apm-server", "uuid": "1f9397f6-2eec-4c16-81f0-748a69f472fb"}}}
2020-02-11T12:08:14.290+0530 INFO [beat] instance/beat.go:950 Build info {"system_info": {"build": {"commit": "0683f8094f42fecedd54cee09c0991febd8ce59a", "libbeat": "7.5.2", "time": "2020-01-15T10:21:07.000Z", "version": "7.5.2"}}}
2020-02-11T12:08:14.290+0530 INFO [beat] instance/beat.go:953 Go runtime info {"system_info": {"go": {"os":"linux","arch":"386","max_procs":4,"version":"go1.12.12"}}}
2020-02-11T12:08:14.291+0530 INFO [beat] instance/beat.go:957 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-01-19T16:58:15+05:30","containerized":false,"name":"CDCUJENUATAPP1","ip":["127.0.0.1/8","::1/128","10.40.17.64/22","fe80::250:56ff:fe87:5864/64"],"kernel_version":"2.6.32-754.25.1.el6.x86_64","mac":["00:50:56:87:58:64"],"os":{"family":"redhat","platform":"redhat","name":"Red","version":"6.10 (Santiago)","major":6,"minor":10,"patch":0,"codename":"Santiago"},"timezone":"IST","timezone_offset_sec":19800,"id":"e88e7ac32c29b1ba4542078000000015"}}}
2020-02-11T12:08:14.292+0530 INFO [beat] instance/beat.go:986 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":null,"effective":null,"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63"],"ambient":null}, "cwd": "/utxciserver/Elastic/elastic_7.5.2/apm-server-7.5.2-linux-x86", "exe": "/utxciserver/Elastic/elastic_7.5.2/apm-server-7.5.2-linux-x86/apm-server", "name": "apm-server", "pid": 5467, "ppid": 28902, "seccomp": {"mode":""}, "start_time": "2020-02-11T12:08:10.440+0530"}}}
2020-02-11T12:08:14.292+0530 INFO instance/beat.go:297 Setup Beat: apm-server; Version: 7.5.2
2020-02-11T12:08:14.293+0530 INFO elasticsearch/client.go:171 Elasticsearch url: http://localhost:9200
2020-02-11T12:08:14.326+0530 INFO [publisher] pipeline/module.go:97 Beat name: CDCUJENUATAPP1
2020-02-11T12:08:14.328+0530 INFO [beater] beater/beater.go:124 Registering pipeline callback
2020-02-11T12:08:14.341+0530 INFO instance/beat.go:429 apm-server start running.
2020-02-11T12:08:14.341+0530 INFO [beater] beater/beater.go:259 self instrumentation is disabled
2020-02-11T12:08:14.343+0530 INFO [handler] api/mux.go:93 Path / added to request handler
2020-02-11T12:08:14.344+0530 INFO [handler] api/mux.go:93 Path /assets/v1/sourcemaps added to request handler
2020-02-11T12:08:14.344+0530 INFO [agentcfg] agentcfg/cache.go:38 Cache creation with expiration 30s.
2020-02-11T12:08:14.349+0530 INFO [handler] api/mux.go:93 Path /config/v1/agents added to request handler
2020-02-11T12:08:14.349+0530 INFO [agentcfg] agentcfg/cache.go:38 Cache creation with expiration 30s.
2020-02-11T12:08:14.349+0530 INFO [handler] api/mux.go:93 Path /config/v1/rum/agents added to request handler
2020-02-11T12:08:14.349+0530 INFO [handler] api/mux.go:93 Path /intake/v2/rum/events added to request handler
2020-02-11T12:08:14.349+0530 INFO [handler] api/mux.go:93 Path /intake/v2/events added to request handler
2020-02-11T12:08:14.350+0530 INFO [beater] beater/server.go:79 Starting apm-server [0683f8094f42fecedd54cee09c0991febd8ce59a built 2020-01-15 10:21:07 +0000 UTC]. Hit CTRL-C to stop it.
2020-02-11T12:08:14.374+0530 INFO [beater] beater/server.go:80 Listening on: 10.40.17.64:8200
2020-02-11T12:08:14.374+0530 INFO [beater] beater/server.go:91 RUM endpoints disabled.
2020-02-11T12:08:14.374+0530 INFO [beater] beater/server.go:106 SSL disabled.
2020-02-11T12:08:14.397+0530 INFO [request] middleware/log_middleware.go:76 request ok {"request_id": "f7e438e2-0ec8-4317-9dce-15ce2ef66876", "method": "GET", "URL": "/", "content_length": 0, "remote_address": "10.40.17.64", "user-agent": "", "response_code": 200}
2020-02-11T12:08:14.397+0530 INFO [onboarding] beater/onboarding.go:32 Publishing onboarding document
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8b8464c]

goroutine 100 [running]:
runtime/internal/atomic.Xadd64(0xbd2404c, 0x1, 0x0, 0x0, 0x3)
/usr/local/go/src/runtime/internal/atomic/asm_386.s:105 +0xc
github.com/elastic/apm-server/vendor/github.com/elastic/beats/libbeat/common/atomic.(*Uint64).Add(...)
/go/src/github.com/elastic/apm-server/vendor/github.com/elastic/beats/libbeat/common/atomic/atomic.go:83
github.com/elastic/apm-server/vendor/github.com/elastic/beats/libbeat/common/atomic.(*Uint64).Inc(...)
/go/src/github.com/elastic/apm-server/vendor/github.com/elastic/beats/libbeat/common/atomic/atomic.go:85
github.com/elastic/apm-server/vendor/github.com/elastic/beats/libbeat/publisher/pipeline.(*waitACK).addEvent(0xbd24030, 0x97b82878, 0xbf88b0d5, 0x7149bf4a, 0x0, 0xb6f6e80, 0xc01e080, 0xc01e9e0, 0x0, 0x0, ...)
/go/src/github.com/elastic/apm-server/vendor/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:462 +0x70
github.com/elastic/apm-server/vendor/github.com/elastic/beats/libbeat/publisher/pipeline.(*client).publish(0xbcc8410, 0x97b82878, 0xbf88b0d5, 0x7149bf4a, 0x0, 0xb6f6e80, 0x0, 0xc01e9e0, 0x0, 0x0, ...)
/go/src/github.com/elastic/apm-server/vendor/github.com/elastic/beats/libbeat/publisher/pipeline/client.go:102 +0x113
github.com/elastic/apm-server/vendor/github.com/elastic/beats/libbeat/publisher/pipeline.(*client).Publish(0xbcc8410, 0x97b82878, 0xbf88b0d5, 0x7149bf4a, 0x0, 0xb6f6e80, 0x0, 0xc01e9e0, 0x0, 0x0, ...)
/go/src/github.com/elastic/apm-server/vendor/github.com/elastic/beats/libbeat/publisher/pipeline/client.go:68 +0x65
github.com/elastic/apm-server/beater.notifyListening(0xbea2e70, 0xc002550)
/go/src/github.com/elastic/apm-server/beater/onboarding.go:40 +0x2a6
created by github.com/elastic/apm-server/beater.(*beater).Run
/go/src/github.com/elastic/apm-server/beater/beater.go:215 +0x73f

Previously I installed APM server version 7.5.0 which gets launched properly.

Please let me know if additional info is required.

2

Hey @shashwatsharma, thanks for reporting this. Can you provide more information to help us track this down? What distribution/version are you running on? Have you customized your configuration apm-server.yml?

3

Hi @gil thanks for your reply..

I am running the Elastic Kibana and APM on Red Hat Enterprise Linux Server release 6.10 (Santiago).
yes I have customized my apm-server.yml . Below is the conf. where I changed

################################ APM Server ################################
apm-server:

Defines the host and port the server is listening on. Use "unix:/path/to.sock" to listen on a unix domain socket.

host: "10.40.17.64:8200"

#---------------------------- APM Server - Agent Configuration ----------------------------

kibana:
# For APM Agent configuration in Kibana, enabled must be true.
enabled: true

# Scheme and port can be left out and will be set to the default (`http` and `5601`).
# In case you specify an additional path, the scheme is required: `http://localhost:5601/path`.
# IPv6 addresses should always be defined as: `https://[2001:db8::1]:5601`.
host: "10.40.17.64:5601"

#-------------------------- Elasticsearch output --------------------------
output.elasticsearch:

Array of hosts to connect to.

Scheme and port can be left out and will be set to the default (http and 9200).

In case you specify and additional path, the scheme is required: http://localhost:9200/path.

IPv6 addresses should always be defined as: https://[2001:db8::1]:9200.

hosts: ["http://10.40.17.64:9200"]

.

One more issue.. I tried with apm server version 7.5.0 , it worked and I have installed APM JAVA agent on app server .

But I am not able to see any transaction in transaction pane although I am getting data in JVM pane.

Please help.

4

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.