Hi all,
having an issue starting an apm-server on an ubuntu vm. See details below.
Kibana version: 7.4.0
Elasticsearch version: 7.4.0
APM Server version: 7.4.0
APM Agent language and version: N/A
Browser version: Version 83.0.4103.97 (Official Build) (64-bit)
Original install method (e.g. download page, yum, deb, from source, etc.) and version: deb
Fresh install or upgraded from other version? Fresh install
Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc. - None of the above
Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
When starting the apm-server service (service apm-server start) the server starts up but then exits straight away with a generic error on a missing path.
Can't really understand which path it's referring to. I didn't change the path config in apm-server.yml so using defaults.
Provide logs and/or server output (if relevant): apm-server debug logs below
2020-06-21T22:05:44.269Z INFO instance/beat.go:607 Home path: [/usr/share/apm-server] Config path: [/etc/apm-server] Data path: [/var/lib/apm-server] Logs path: [/var/log/apm-server]
2020-06-21T22:05:44.269Z DEBUG [beat] instance/beat.go:659 Beat metadata path: /var/lib/apm-server/meta.json
2020-06-21T22:05:44.269Z INFO instance/beat.go:615 Beat ID: 7d3ea0e7-866d-4e67-bf4e-54a4958a72c2
2020-06-21T22:05:44.270Z DEBUG [seccomp] seccomp/seccomp.go:117 Loading syscall filter {"seccomp_filter": {"no_new_privs":true,"flag":"tsync","policy":{"default_action":"errno","syscalls":[{"names":["accept","accept4","access","arch_prctl","bind","brk","clock_gettime","clone","close","connect","dup","dup2","epoll_create","epoll_create1","epoll_ctl","epoll_pwait","epoll_wait","exit","exit_group","fchdir","fchmod","fchown","fcntl","fdatasync","flock","fstat","fstatfs","fsync","ftruncate","futex","getcwd","getdents","getdents64","geteuid","getgid","getpeername","getpid","getppid","getrandom","getrlimit","getrusage","getsockname","getsockopt","gettid","gettimeofday","getuid","inotify_add_watch","inotify_init1","inotify_rm_watch","ioctl","kill","listen","lseek","lstat","madvise","mincore","mkdirat","mmap","mprotect","munmap","nanosleep","newfstatat","open","openat","pipe","pipe2","poll","ppoll","pread64","pselect6","pwrite64","read","readlink","readlinkat","recvfrom","recvmmsg","recvmsg","rename","renameat","rt_sigaction","rt_sigprocmask","rt_sigreturn","sched_getaffinity","sched_yield","sendfile","sendmmsg","sendmsg","sendto","set_robust_list","setitimer","setsockopt","shutdown","sigaltstack","socket","splice","stat","statfs","sysinfo","tgkill","time","tkill","uname","unlink","unlinkat","wait4","waitid","write","writev"],"action":"allow"}]}}}
2020-06-21T22:05:44.270Z INFO [seccomp] seccomp/seccomp.go:124 Syscall filter successfully installed
2020-06-21T22:05:44.270Z INFO [beat] instance/beat.go:903 Beat info {"system_info": {"beat": {"path": {"config": "/etc/apm-server", "data": "/var/lib/apm-server", "home": "/usr/share/apm-server", "logs": "/var/log/apm-server"}, "type": "apm-server", "uuid": "7d3ea0e7-866d-4e67-bf4e-54a4958a72c2"}}}
2020-06-21T22:05:44.270Z INFO [beat] instance/beat.go:912 Build info {"system_info": {"build": {"commit": "971d864356e4438bf4a799a1fa052cfd0ce680b4", "libbeat": "7.4.0", "time": "2019-09-27T06:58:27.000Z", "version": "7.4.0"}}}
2020-06-21T22:05:44.270Z INFO [beat] instance/beat.go:915 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.12.9"}}}
2020-06-21T22:05:44.270Z INFO [beat] instance/beat.go:919 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-04-08T06:59:14Z","containerized":false,"name":"lts-logging-node-1","ip":["127.0.0.1/8","::1/128","10.3.0.5/24","fe80::20d:3aff:fe20:52a8/64"],"kernel_version":"3.13.0-137-generic","mac":["00:0d:3a:20:52:a8"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"14.04.5 LTS, Trusty Tahr","major":14,"minor":4,"patch":5,"codename":"trusty"},"timezone":"UTC","timezone_offset_sec":0,"id":"ca359a30ee39eda0d5e4dbf9592dd97e"}}}
2020-06-21T22:05:44.271Z INFO [beat] instance/beat.go:948 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"effective":null,"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"ambient":null}, "cwd": "/", "exe": "/usr/share/apm-server/bin/apm-server", "name": "apm-server", "pid": 130402, "ppid": 130401, "seccomp": {"mode":"filter"}, "start_time": "2020-06-21T22:05:43.890Z"}}}
2020-06-21T22:05:44.271Z INFO instance/beat.go:292 Setup Beat: apm-server; Version: 7.4.0
2020-06-21T22:05:44.271Z DEBUG [beat] instance/beat.go:318 Initializing output plugins
2020-06-21T22:05:44.271Z DEBUG [conditions] conditions/conditions.go:98 New condition contains: map[]
2020-06-21T22:05:44.271Z DEBUG [conditions] conditions/conditions.go:98 New condition contains: map[]
2020-06-21T22:05:44.271Z DEBUG [conditions] conditions/conditions.go:98 New condition contains: map[]
2020-06-21T22:05:44.271Z DEBUG [conditions] conditions/conditions.go:98 New condition contains: map[]
2020-06-21T22:05:44.271Z DEBUG [conditions] conditions/conditions.go:98 New condition contains: map[]
2020-06-21T22:05:44.271Z DEBUG [conditions] conditions/conditions.go:98 New condition contains: map[]
2020-06-21T22:05:44.271Z DEBUG [conditions] conditions/conditions.go:98 New condition contains: map[]
2020-06-21T22:05:44.272Z DEBUG [conditions] conditions/conditions.go:98 New condition contains: map[]
2020-06-21T22:05:44.272Z DEBUG [conditions] conditions/conditions.go:98 New condition contains: map[]
2020-06-21T22:05:44.272Z DEBUG [conditions] conditions/conditions.go:98 New condition contains: map[]
2020-06-21T22:05:44.272Z DEBUG [conditions] conditions/conditions.go:98 New condition contains: map[]
2020-06-21T22:05:44.272Z DEBUG [conditions] conditions/conditions.go:98 New condition contains: map[]
2020-06-21T22:05:44.272Z INFO elasticsearch/client.go:170 Elasticsearch url: http://10.3.0.8:9200
2020-06-21T22:05:44.272Z INFO elasticsearch/client.go:170 Elasticsearch url: http://10.3.0.9:9200
2020-06-21T22:05:44.272Z INFO elasticsearch/client.go:170 Elasticsearch url: http://10.3.0.10:9200
2020-06-21T22:05:44.272Z DEBUG [publisher] pipeline/consumer.go:137 start pipeline event consumer
2020-06-21T22:05:44.272Z INFO [publisher] pipeline/module.go:97 Beat name: lts-logging-node-1
2020-06-21T22:05:44.273Z INFO [beater] beater/beater.go:124 Registering pipeline callback
2020-06-21T22:05:44.273Z INFO instance/beat.go:422 apm-server start running.
2020-06-21T22:05:44.273Z INFO [beater] beater/beater.go:259 self instrumentation is disabled
2020-06-21T22:05:44.274Z INFO [beater] beater/beater.go:165 host resolved from localhost:8200 to 127.0.0.1:8200
2020-06-21T22:05:44.274Z INFO [handler] api/mux.go:91 Path / added to request handler
2020-06-21T22:05:44.274Z INFO [handler] api/mux.go:91 Path /assets/v1/sourcemaps added to request handler
2020-06-21T22:05:44.274Z INFO [agentcfg] agentcfg/cache.go:42 Cache creation with default expiration 30s.
2020-06-21T22:05:44.274Z INFO [handler] api/mux.go:91 Path /config/v1/agents added to request handler
2020-06-21T22:05:44.274Z INFO [handler] api/mux.go:91 Path /intake/v2/rum/events added to request handler
2020-06-21T22:05:44.274Z INFO [handler] api/mux.go:91 Path /intake/v2/events added to request handler
2020-06-21T22:05:44.274Z DEBUG [kibana] kibana/connecting_client.go:73 Trying to obtain connection to Kibana.
2020-06-21T22:05:44.274Z INFO kibana/client.go:117 Kibana url: http://127.0.0.1:5601
2020-06-21T22:05:44.275Z INFO [beater] beater/server.go:79 Starting apm-server [971d864356e4438bf4a799a1fa052cfd0ce680b4 built 2019-09-27 06:58:27 +0000 UTC]. Hit CTRL-C to stop it.
2020-06-21T22:05:44.275Z INFO [beater] beater/server.go:80 Listening on: 127.0.0.1:8200
2020-06-21T22:05:44.275Z INFO [beater] beater/server.go:91 RUM endpoints disabled.
2020-06-21T22:05:44.275Z INFO [beater] beater/server.go:100 SSL enabled.
2020-06-21T22:05:44.296Z INFO [kibana] kibana/connecting_client.go:80 Successfully obtained connection to Kibana.
2020-06-21T22:05:49.274Z DEBUG [publisher] pipeline/client.go:162 client: closing acker
2020-06-21T22:05:49.274Z DEBUG [publisher] pipeline/client.go:173 client: done closing acker
2020-06-21T22:05:49.274Z DEBUG [publisher] pipeline/client.go:176 client: cancelled 0 events
2020-06-21T22:05:49.274Z DEBUG [publisher] pipeline/client.go:147 client: wait for acker to finish
2020-06-21T22:05:49.274Z DEBUG [publisher] pipeline/client.go:149 client: acker shut down
2020-06-21T22:05:49.274Z INFO instance/beat.go:432 apm-server stopped.
2020-06-21T22:05:49.306Z ERROR instance/beat.go:878 Exiting: open : no such file or directory