I am trying to run the command
filebeat setup -e
and this is the error I get
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.666Z","log.origin":{"file.name":"instance/beat.go","file.line":783},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.667Z","log.origin":{"file.name":"instance/beat.go","file.line":791},"message":"Beat ID: bdaac60f-fd36-4833-9097-1720b7bac3a7","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.673Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1303},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/etc/filebeat","data":"/var/lib/filebeat","home":"/usr/share/filebeat","logs":"/var/log/filebeat"},"type":"filebeat","uuid":"bdaac60f-fd36-4833-9097-1720b7bac3a7"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.673Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1312},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"480bccf4f0423099bb2c0e672a44c54ecd7a805e","libbeat":"8.10.2","time":"2023-09-18T18:09:06.000Z","version":"8.10.2"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.673Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1315},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":2,"version":"go1.20.7"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.674Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1321},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2023-10-05T02:34:02Z","containerized":false,"name":"suricata","ip":["127.0.0.1","::1","159.223.222.118","10.18.0.6","2a03:b0c0:2:d0::e1f:c001","fe80::ecc5:d6ff:fe62:2951","10.133.0.3","fe80::bcc1:9bff:fed9:8ac6"],"kernel_version":"6.1.0-12-amd64","mac":["ee:c5:d6:62:29:51","be:c1:9b:d9:8a:c6"],"os":{"type":"linux","family":"debian","platform":"debian","name":"Debian GNU/Linux","version":"12 (bookworm)","major":12,"minor":0,"patch":0,"codename":"bookworm"},"timezone":"UTC","timezone_offset_sec":0,"id":"f160b40b6d64a29c1b4c5dea651d6916"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.675Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1350},"message":"Process info","service.name":"filebeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","perfmon","bpf","checkpoint_restore"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","perfmon","bpf","checkpoint_restore"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","perfmon","bpf","checkpoint_restore"],"ambient":null},"cwd":"/home/navjot","exe":"/usr/share/filebeat/bin/filebeat","name":"filebeat","pid":1196,"ppid":1195,"seccomp":{"mode":"disabled","no_new_privs":false},"start_time":"2023-10-05T02:55:42.540Z"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.675Z","log.origin":{"file.name":"instance/beat.go","file.line":329},"message":"Setup Beat: filebeat; Version: 8.10.2","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2023-10-05T02:55:42.677Z","log.logger":"cfgwarn","log.origin":{"file.name":"tlscommon/config.go","file.line":102},"message":"DEPRECATED: Treating the CommonName field on X.509 certificates as a host name when no Subject Alternative Names are present is going to be removed. Please update your certificates if needed. Will be removed in version: 8.0.0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.678Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":108},"message":"elasticsearch url: https://10.133.0.2:9200","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.678Z","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":105},"message":"Beat name: suricata","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.678Z","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":135},"message":"Enabled modules/filesets: ","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.679Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":108},"message":"elasticsearch url: https://10.133.0.2:9200","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2023-10-05T02:55:42.690Z","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/provider_aws_ec2.go","file.line":91},"message":"error fetching EC2 Identity Document: operation error ec2imds: GetInstanceIdentityDocument, http response error StatusCode: 404, request to EC2 IMDS failed.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.690Z","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/add_cloud_metadata.go","file.line":104},"message":"add_cloud_metadata: hosting provider type detected as digitalocean, metadata={\"cloud\":{\"instance\":{\"id\":\"377586531\"},\"provider\":\"digitalocean\",\"region\":\"ams3\",\"service\":{\"name\":\"Droplets\"}}}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.710Z","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":179},"message":"'ca_trusted_fingerprint' set, looking for matching fingerprints","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.710Z","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":199},"message":"CA certificate matching 'ca_trusted_fingerprint' found, adding it to 'certificate_authorities'","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.720Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":291},"message":"Attempting to connect to Elasticsearch version 8.10.2","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.720Z","log.logger":"index-management","log.origin":{"file.name":"idxmgmt/std.go","file.line":230},"message":"Auto ILM enable success.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.724Z","log.logger":"index-management.ilm","log.origin":{"file.name":"ilm/std.go","file.line":128},"message":"ILM policy filebeat successfully created.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.724Z","log.logger":"index-management","log.origin":{"file.name":"idxmgmt/std.go","file.line":365},"message":"Set settings.index.lifecycle.name in template to {filebeat {\"policy\":{\"phases\":{\"hot\":{\"actions\":{\"rollover\":{\"max_age\":\"30d\",\"max_primary_shard_size\":\"50gb\"}}}}}}} as ILM is enabled.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:42.739Z","log.logger":"template","log.origin":{"file.name":"template/load.go","file.line":245},"message":"Existing template will be overwritten, as overwrite is enabled.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:44.464Z","log.logger":"template_loader","log.origin":{"file.name":"template/load.go","file.line":159},"message":"Try loading template filebeat-8.10.2 to Elasticsearch","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:44.565Z","log.logger":"template_loader","log.origin":{"file.name":"template/load.go","file.line":127},"message":"Template with name \"filebeat-8.10.2\" loaded.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:44.569Z","log.logger":"template_loader","log.origin":{"file.name":"template/load.go","file.line":143},"message":"Data stream with name \"filebeat-8.10.2\" already exists.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-05T02:55:44.569Z","log.logger":"index-management","log.origin":{"file.name":"idxmgmt/std.go","file.line":266},"message":"Loaded index template.","service.name":"filebeat","ecs.version":"1.6.0"}
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
{"log.level":"info","@timestamp":"2023-10-05T02:55:44.570Z","log.logger":"kibana","log.origin":{"file.name":"kibana/client.go","file.line":183},"message":"Kibana url: http://10.133.0.2:5601","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-10-05T02:55:44.580Z","log.origin":{"file.name":"instance/beat.go","file.line":1278},"message":"Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to http://10.133.0.2:5601/api/status fails: status=503. Response: {\"status\":{\"overall\":{\"level\":\"unavailable\"}}}","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to http://10.133.0.2:5601/api/status fails: status=503. Response: {"status":{"overall":{"level":"unavailable"}}}
This is a brand new installation and Kibana and Elasticsearch are both hosted on the same server while Filebeat is on a different server. I can connect to Elasticsearch from the filebeat server but not Kibana. Please advise.