Hi,
We are running a 3 node elastic cluster to index logs from a firewall.
For monitoring the cluster health, we are trying to collect the elastic cluster logs through filebeat and visualize these logs in kibana. For this we installed filebeat in the one of the nodes within the cluster.
The configuration of filebeat.yml is as follows:
filebeat.inputs:
- type: filestream
id: my-filestream-id
enabled: true
paths:
- /var/log/*.log
# ---------------------------- Elasticsearch Output -----------------------------
output.elasticsearch:
hosts: ["https://192.168.1.6:9200","https://192.168.1.20:9200","https://192.168.1.21:9200"]
preset: balanced
protocol: "https"
username: "username"
password: "PASSWORD"
ssl:
enabled: true
#this will be replaced using secret keystore once it works
ca_trusted_fingerprint: "22d0ddcf89079acec7ef398d4cb658c2ec560ece393f03ffede55c84692f73e1"
certificate_authorities: "/etc/elasticsearch/certs/elastic-stack-ca.pem"
# =================================== Kibana ===================================
setup.kibana:
host: "https://192.168.1.6:5601"
username: "username"
password: "PASSWORD"
ssl:
enabled: true
certificate_authorities: "/etc/kibana/elastic-stack-ca.pem"
certificate: "/etc/kibana/kibana-server.pem"
key: "/etc/kibana/kibana-server.key"
We also enabled the elasticsearch module in modules.d in Filebeat and the configuration of elasticsearch.yml is as follows:
- module: elasticsearch
# Server log
server:
enabled: true
var.paths:
- /var/log/elasticsearch/*.json
- /var/log/elasticsearch/*_server.json
While settingup Filebeat by running following command: " filebeat setup -e " we are getting the errors mentioned below:
{"log.level":"error","@timestamp":"2024-04-23T16:15:54.092+0530","log.logger":"add_cloud_metadata","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/processors/add_cloud_metadata.(*addCloudMetadata).fetchMetadata","file.name":"add_cloud_metadata/providers.go","file.line":173},"message":"add_cloud_metadata: received error failed requesting azure metadata: Get \"http://169.254.169.254/metadata/instance/compute?api-version=2021-02-01\": dial tcp 169.254.169.254:80: i/o timeout","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2024-04-23T16:17:21.102+0530","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.NewConnectedClient","file.name":"eslegclient/connection.go","file.line":252},"message":"error connecting to Elasticsearch at https://192.168.1.6:9200: Get \"https://192.168.1.6:9200\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2024-04-23T16:17:21.102+0530","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/elastic-agent-libs/transport/httpcommon.(*HTTPTransportSettings).RoundTripper.LoggingDialer.func2","file.name":"transport/logging.go","file.line":38},"message":"Error dialing dial tcp 192.168.1.6:9200: i/o timeout","service.name":"filebeat","network":"tcp","address":"192.168.1.6:9200","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2024-04-23T16:17:21.904+0530","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.handleError","file.name":"instance/beat.go","file.line":1340},"message":"Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to https://192.168.1.6:5601/api/status fails: fail to execute the HTTP GET request: Get \"https://192.168.1.6:5601/api/status\": x509: certificate signed by unknown authority (status=0). Response: ","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to https://192.168.1.6:5601/api/status fails: fail to execute the HTTP GET request: Get "https://192.168.1.6:5601/api/status": x509: certificate signed by unknown authority (status=0). Response:
We are using xpack security with self signed certificates. The error appears to be related to the use of a self signed certificate.
How can this be resolved?