I am trying to use the kubernetes metricbeat module inorder to monitor pod status with Elastic search and Kibana.
However when I run metric beat as docker, I get the below error.
Component Image version:
2018/05/07 08:44:05.013171 asm_amd64.s:515: DBG Stopped metricSetWrapper[module=kubernetes, name=container, host=some.IP:10250]
2018/05/07 08:44:05.013200 log.go:175: ERR recovered from panic while fetching 'kubernetes/container' for host 'some.IP:10250'. Recovering, but please report this: runtime error: invalid memory address or nil pointer dereference.
2018/05/07 08:44:05.013291 log.go:176: ERR Stacktrace: goroutine 395 [running]:
runtime/debug.Stack(0x2a67e9d, 0x2b, 0xc4206196f8)
/usr/local/go/src/runtime/debug/stack.go:24 +0x79
github.com/elastic/beats/libbeat/logp.Recover(0xc420596240, 0x58)
/go/src/github.com/elastic/beats/libbeat/logp/log.go:176 +0x12f
panic(0x26f4540, 0x38c9c50)
/usr/local/go/src/runtime/panic.go:489 +0x2cf
github.com/elastic/beats/metricbeat/helper.(*HTTP).FetchResponse(0x0, 0xc4206ebd00, 0x1743659, 0xc42007ab40)
/go/src/github.com/elastic/beats/metricbeat/helper/http.go:77 +0x3a
github.com/elastic/beats/metricbeat/helper.(*HTTP).FetchContent(0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/elastic/beats/metricbeat/helper/http.go:116 +0x7c
github.com/elastic/beats/metricbeat/module/kubernetes/container.(*MetricSet).Fetch(0xc4202359e0, 0x2, 0xc4205829e0, 0xc420626960, 0x2, 0xc4206ebbd0)
/go/src/github.com/elastic/beats/metricbeat/module/kubernetes/container/container.go:55 +0x36
github.com/elastic/beats/metricbeat/mb/module.(*metricSetWrapper).multiEventFetch(0xc42054f2c0, 0x7f3cd1d5d2f0, 0xc4202359e0, 0x38f7be0, 0xc4201161b0)
/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:238 +0x4f
github.com/elastic/beats/metricbeat/mb/module.(*metricSetWrapper).fetch(0xc42054f2c0, 0x38f7be0, 0xc4201161b0)
/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:222 +0x118
github.com/elastic/beats/metricbeat/mb/module.(*metricSetWrapper).startPeriodicFetching(0xc42054f2c0, 0x38f7be0, 0xc4201161b0)
/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:199 +0x5d
github.com/elastic/beats/metricbeat/mb/module.(*metricSetWrapper).run(0xc42054f2c0, 0xc4206d5ec0, 0xc4201e5aa0)
/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:186 +0x5b8
github.com/elastic/beats/metricbeat/mb/module.(*Wrapper).Start.func1(0xc4201f7e20, 0xc4206d5ec0, 0xc4201e5aa0, 0xc42054f2c0)
/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:110 +0xbd
created by github.com/elastic/beats/metricbeat/mb/module.(*Wrapper).Start
/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:111 +0x14e
2018/05/07 08:44:05.252913 wrapper.go:172: DBG Starting metricSetWrapper[module=system, name=process_summary, host=]
2018/05/07 08:44:05.253487 processor.go:262: DBG Publish event: {
"@timestamp": "2018-05-07T08:44:05.252Z",
module.d/kubernetes.yml:
# Node metrics, from kubelet:
- module: kubernetes
metricsets:
- node
- system
- pod
- container
- volume
period: 10s
hosts: ["https://some.ip:10250"]
ssl.certificate_authorities: ["/etc/origin/node/ca.crt"]
ssl.certificate: "/etc/origin/node/system:node:some.hostname.crt"
ssl.key: "/etc/origin/node/system:node:some.hostname.key"
# State metrics from kube-state-metrics service:
- module: kubernetes
enabled: false
metricsets:
- state_node
- state_deployment
- state_replicaset
- state_pod
- state_container
period: 10s
hosts: ["kube-state-metrics:8080"]
# Kubernetes events
- module: kubernetes
enabled: true
metricsets:
- event