metricbeat version: 7.17.20
Metricbeat stopped sending process information 2 min after the restart. After that, it stops sending 'system/process' events and shows the following error message in the log file:
{"level":"info","timestamp":"2024-06-17T16:26:49.957Z","caller":"template/load.go:110","message":"Template \"metricbeat-7.17.20\" already exists and will not be overwritten."}
{"level":"info","timestamp":"2024-06-17T16:26:49.957Z","logger":"index-management","caller":"idxmgmt/std.go:296","message":"Loaded index template."}
{"level":"info","timestamp":"2024-06-17T16:26:49.959Z","logger":"index-management.ilm","caller":"ilm/std.go:126","message":"Index Alias metricbeat-7.17.20 exists already."}
{"level":"info","timestamp":"2024-06-17T16:26:49.959Z","logger":"publisher_pipeline_output","caller":"pipeline/output.go:151","message":"Connection to backoff(elasticsearch(http://10.0.38.172:10002)) established"}
{"level":"error","timestamp":"2024-06-17T16:28:19.814Z","caller":"runtime/iface.go:263","message":"recovered from panic while fetching 'system/process' for host ''. Recovering, but please report this.","panic":"interface conversion: cgroup.CGStats is nil, not *cgroup.StatsV2","stack":"github.com/elastic/beats/v7/libbeat/logp.Recover\n\t/go/src/github.com/elastic/beats/libbeat/logp/global.go:102\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:914\nruntime.panicdottypeE\n\t/usr/local/go/src/runtime/iface.go:263\nruntime.panicdottypeI\n\t/usr/local/go/src/runtime/iface.go:273\ngithub.com/elastic/beats/v7/libbeat/metric/system/cgroup.(*StatsV2).FillPercentages\n\t/go/src/github.com/elastic/beats/libbeat/metric/system/cgroup/cgstats.go:122\ngithub.com/elastic/beats/v7/libbeat/metric/system/process.(*Stats).getSingleProcess\n\t/go/src/github.com/elastic/beats/libbeat/metric/system/process/process.go:559\ngithub.com/elastic/beats/v7/libbeat/metric/system/process.(*Stats).Get\n\t/go/src/github.com/elastic/beats/libbeat/metric/system/process/process.go:480\ngithub.com/elastic/beats/v7/metricbeat/module/system/process.(*MetricSet).Fetch\n\t/go/src/github.com/elastic/beats/metricbeat/module/system/process/process.go:114\ngithub.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).fetch\n\t/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:256\ngithub.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).startPeriodicFetching\n\t/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:234\ngithub.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).run\n\t/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:208\ngithub.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func1\n\t/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:147"}
/etc/metricbeat/metricbeat.yml
# Ansible managed
################### metricbeat Configuration #########################
############################# metricbeat ######################################
metricbeat.modules:
- metricsets:
- cpu
- load
- memory
- network
- process
- diskio
- service
- users
module: system
period: 10s
process.include_top_n:
by_cpu: 10
by_memory: 10
###############################################################################
############################# Libbeat Config ##################################
# Base config file used by all other beats for using libbeat features
############################# Output ##########################################
output:
elasticsearch:
hosts:
- http://10.0.38.171:10002
- http://10.0.38.172:10002
############################# Logging #########################################
logging:
files:
keepfiles: '3'
name: metricbeat
path: /var/log/metricbeat
permissions: '0644'
json: true
level: info
metrics.enabled: false
to_files: true
to_syslog: false
# yum list metricbeat
This system is receiving updates from OSMS server.
Last metadata expiration check: 0:20:53 ago on Mon 17 Jun 2024 04:26:36 PM GMT.
Installed Packages
metricbeat.x86_64 7.17.20-1 @beats
[
# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.9 (Ootpa)
The server is running in Oracle OCI Cloud.