Metricbeat Output to Secure Elasticsearch Error

Hi,

I'm using ELK Stack 8.0, which is Elasticsearch running in secured mode.
I'm having the problem to output my metricbeat to Elasticsearch.
Below is the error when I run the command "metricbeat setup -e"

{"log.level":"info","@timestamp":"2022-03-28T18:43:55.067+0800","log.origin":{"file.name":"instance/beat.go","file.line":669},"message":"Home path: [/usr/share/metricbeat] Config path: [/etc/metricbeat] Data path: [/var/lib/metricbeat] Logs path: [/var/log/metricbeat]","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-03-28T18:43:55.068+0800","log.origin":{"file.name":"instance/beat.go","file.line":677},"message":"Beat ID: eb5f7e0b-7f3d-42d4-9c48-2f63ff0aefc8","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-03-28T18:43:58.070+0800","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/provider_aws_ec2.go","file.line":80},"message":"read token request for getting IMDSv2 token returns empty: Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used.","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-03-28T18:43:58.071+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1047},"message":"Beat info","service.name":"metricbeat","system_info":{"beat":{"path":{"config":"/etc/metricbeat","data":"/var/lib/metricbeat","home":"/usr/share/metricbeat","logs":"/var/log/metricbeat"},"type":"metricbeat","uuid":"eb5f7e0b-7f3d-42d4-9c48-2f63ff0aefc8"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-03-28T18:43:58.071+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1056},"message":"Build info","service.name":"metricbeat","system_info":{"build":{"commit":"7f30bb31a4a532c865161efbbdadd012323b04c5","libbeat":"8.1.1","time":"2022-03-17T23:25:44.000Z","version":"8.1.1"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-03-28T18:43:58.071+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1059},"message":"Go runtime info","service.name":"metricbeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":4,"version":"go1.17.6"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-03-28T18:43:58.072+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1063},"message":"Host info","service.name":"metricbeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2022-03-25T18:46:40+08:00","containerized":false,"name":"z3lynx-kbn-01","ip":["127.0.0.1/8","::1/128","10.10.10.81/24"],"kernel_version":"4.18.0-373.el8.x86_64","mac":["00:50:56:b6:8b:50"],"os":{"type":"linux","family":"redhat","platform":"centos","name":"CentOS Stream","version":"8","major":8,"minor":0,"patch":0},"timezone":"+08","timezone_offset_sec":28800,"id":"2b4df34ad3d943a89fc2759232e53922"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-03-28T18:43:58.072+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1092},"message":"Process info","service.name":"metricbeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40"],"ambient":null},"cwd":"/tmp","exe":"/usr/share/metricbeat/bin/metricbeat","name":"metricbeat","pid":5359,"ppid":4486,"seccomp":{"mode":"disabled","no_new_privs":false},"start_time":"2022-03-28T18:43:54.220+0800"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-03-28T18:43:58.072+0800","log.origin":{"file.name":"instance/beat.go","file.line":323},"message":"Setup Beat: metricbeat; Version: 8.1.1","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-03-28T18:43:58.073+0800","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: https://10.10.10.61:9200","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-03-28T18:43:58.074+0800","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: z3lynx-kbn-01","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-03-28T18:43:58.096+0800","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: https://10.10.10.61:9200","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-03-28T18:43:58.125+0800","log.logger":"esclientleg","log.origin":{"file.name":"transport/logging.go","file.line":37},"message":"Error dialing x509: certificate signed by unknown authority","service.name":"metricbeat","network":"tcp","address":"10.10.10.61:9200","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-03-28T18:43:58.125+0800","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":231},"message":"error connecting to Elasticsearch at https://10.10.10.61:9200: Get \"https://10.10.10.61:9200\": x509: certificate signed by unknown authority","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-03-28T18:43:58.125+0800","log.origin":{"file.name":"instance/beat.go","file.line":1022},"message":"Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at https://10.10.10.61:9200: Get \"https://10.10.10.61:9200\": x509: certificate signed by unknown authority]","service.name":"metricbeat","ecs.version":"1.6.0"}
Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at https://10.10.10.61:9200: Get "https://10.10.10.61:9200": x509: certificate signed by unknown authority]

My metricbeat.yml config,

setup.kibana:
  host: "https://10.10.10.81:5601"

output.elasticsearch:
  hosts: ["https://10.10.10.61:9200"]
  protocol: "https"
  username: "elastic"
  password: "my-password"

My kibana-xpack.yml config,

- module: kibana
  metricsets:
    - stats
  xpack.enabled: true
  period: 10s
  hosts: ["https://localhost:5601"]
  #basepath: ""
  username: "elastic"
  password: "my-password"

Anyone can assist me on that?
Appreciate for your help.

Hi there,

See Secure communication with Elasticsearch | Metricbeat Reference [8.1] | Elastic.
What you need is to add ssl.certificate_authorities: to your output.elasticsearch part of the configuration. The value should be the path to the CA certificate for the HTTP layer that Elasticsearch generates for you, and which is path/where/elasticsearch/is/installed/config/certs/http_ca.crt

1 Like

Hi,
It is working now.
Thanks a lot...

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.