Filebeat fails to start with add_kubernetes_metadata

[Adam_Wong]

When starting filebeat using add_kubernetes_metadata, the beat will report an error like this and stop:

2019-07-02T19:32:29.785+0800	INFO	instance/beat.go:279	Setup Beat: filebeat; Version: 7.0.0-alpha1
2019-07-02T19:32:29.785+0800	INFO	kubernetes/util.go:86	kubernetes: Using pod name filebeat-zzvs8 and namespace kube-system to discover kubernetes node
2019-07-02T19:32:31.253+0800	ERROR	kubernetes/util.go:90	kubernetes: Querying for pod failed with error: %!(EXTRA string=performing request: Get https://1.1.1.1:443/api/v1/namespaces/kube-system/pods/filebeat-zzvs8: x509: certificate signed by unknown authority)
2019-07-02T19:32:31.254+0800	INFO	kubernetes/watcher.go:180	kubernetes: Performing a resource sync for *v1.PodList
2019-07-02T19:32:31.711+0800	ERROR	kubernetes/watcher.go:183	kubernetes: Performing a resource sync err performing request: Get https://1.1.1.1:443/api/v1/pods?fieldSelector=spec.nodeName%3Dlocalhost&resourceVersion=0: x509: certificate signed by unknown authority for *v1.PodList
2019-07-02T19:32:31.711+0800	INFO	instance/beat.go:340	filebeat stopped.
2019-07-02T19:32:31.711+0800	ERROR	instance/beat.go:758	Exiting: error initializing publisher: error initializing processors: performing request: Get https://1.1.1.1:443/api/v1/pods?fieldSelector=spec.nodeName%3Dlocalhost&resourceVersion=0: x509: certificate signed by unknown authority
Exiting: error initializing publisher: error initializing processors: performing request: Get https://1.1.1.1:443/api/v1/pods?fieldSelector=spec.nodeName%3Dlocalhost&resourceVersion=0: x509: certificate signed by unknown authority

and add_kubernetes_metadata config is:

  - add_kubernetes_metadata:
    enabled: true
    in_cluster: true

this is the version I am using.But it works in another cluster.

[faec (Fae Charlton)]

It looks like the problem isn't your filebeat config but rather your system's SSL config -- it doesn't recognize your kubernetes server as coming from a trusted source. Does your kubernetes setup perhaps use a self-signed certificate? Here is a link on adding certificate authorities that shows a way to update it from kubernetes, you may want to compare your certificates with the cluster that works and import the root certificate from there.