At the same time as i have tried to change my elasticsearch.yml fle a little bit to accommodate http.ssl, since then i have not been able to connect the beats to my elasticsearch, and i am also going to post my elasticsearch.yml file down below :
└─# service elasticsearch start
Job for elasticsearch.service failed because the control process exited with error code.
See "systemctl status elasticsearch.service" and "journalctl -xeu elasticsearch.service" for details.
Job for elasticsearch.service failed because the control process exited with error code.
┌──(root㉿kali)-[/home/kali]
└─# service elasticsearch status
× elasticsearch.service - LSB: Starts elasticsearch
Loaded: loaded (/etc/init.d/elasticsearch; generated)
Active: failed (Result: exit-code) since Tue 2023-05-30 10:37:40 EDT; 11s ago
Docs: man:systemd-sysv-generator(8)
Process: 334444 ExecStart=/etc/init.d/elasticsearch start (code=exited, status=1/FAILURE)
CPU: 49ms
May 30 10:37:40 kali systemd[1]: Starting elasticsearch.service - LSB: Starts elasticsearch...
May 30 10:37:40 kali elasticsearch[334444]: The elasticsearch startup script does not exists or it is not executable, tried: /usr/share/elasticsearch/bin/elasticsearch
May 30 10:37:40 kali systemd[1]: elasticsearch.service: Control process exited, code=exited, status=1/FAILURE
May 30 10:37:40 kali systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
May 30 10:37:40 kali systemd[1]: Failed to start elasticsearch.service - LSB: Starts elasticsearch.
┌──(root㉿kali)-[/home/kali]
└─# service elasticsearch start
Job for elasticsearch.service failed because the control process exited with error code.
See "systemctl status elasticsearch.service" and "journalctl -xeu elasticsearch.service" for details.
From our Linux Machine :
└─# auditbeat setup -e
2023-05-30T10:36:15.546-0400 INFO instance/beat.go:698 Home path: [/usr/share/auditbeat] Config path: [/etc/auditbeat] Data path: [/var/lib/auditbeat] Logs path: [/var/log/auditbeat] Hostfs Path: [/]
2023-05-30T10:36:15.625-0400 INFO instance/beat.go:706 Beat ID: e68758c6-798e-4329-8ef5-33017a1c862b
2023-05-30T10:36:18.649-0400 WARN [add_cloud_metadata] add_cloud_metadata/provider_aws_ec2.go:79 read token request for getting IMDSv2 token returns empty: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used.
2023-05-30T10:36:18.892-0400 INFO [beat] instance/beat.go:1052 Beat info {"system_info": {"beat": {"path": {"config": "/etc/auditbeat", "data": "/var/lib/auditbeat", "home": "/usr/share/auditbeat", "logs": "/var/log/auditbeat"}, "type": "auditbeat", "uuid": "e68758c6-798e-4329-8ef5-33017a1c862b"}}}
2023-05-30T10:36:18.893-0400 INFO [beat] instance/beat.go:1061 Build info {"system_info": {"build": {"commit": "78a342312954e587301b653093954ff7ee4d4f2b", "libbeat": "7.17.10", "time": "2023-04-23T08:09:56.000Z", "version": "7.17.10"}}}
2023-05-30T10:36:18.893-0400 INFO [beat] instance/beat.go:1064 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.19.7"}}}
2023-05-30T10:36:18.893-0400 INFO [beat] instance/beat.go:1070 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2023-05-29T17:55:35-04:00","containerized":false,"name":"kali","ip":["127.0.0.1","::1","192.168.2.18","fe80::a00:27ff:feb1:9d67","fe80::a00:27ff:feda:76e8","172.17.0.1","172.20.0.1","172.18.0.1"],"kernel_version":"6.1.0-kali7-amd64","mac":["08:00:27:b1:9d:67","08:00:27:da:76:e8","02:42:d3:b9:da:1e","02:42:30:56:51:a1","02:42:a1:0b:66:4d"],"os":{"type":"linux","family":"","platform":"kali","name":"Kali GNU/Linux","version":"2023.1","major":2023,"minor":1,"patch":0,"codename":"kali-rolling"},"timezone":"EDT","timezone_offset_sec":-14400,"id":"3095ed18a81a4f50ba21f01bf6332087"}}}
2023-05-30T10:36:18.894-0400 INFO [beat] instance/beat.go:1099 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40"],"ambient":null}, "cwd": "/home/kali", "exe": "/usr/share/auditbeat/bin/auditbeat", "name": "auditbeat", "pid": 333679, "ppid": 290806, "seccomp": {"mode":"disabled","no_new_privs":false}, "start_time": "2023-05-30T10:36:08.940-0400"}}}
2023-05-30T10:36:18.894-0400 INFO instance/beat.go:292 Setup Beat: auditbeat; Version: 7.17.10
2023-05-30T10:36:18.894-0400 INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'auditbeat-7.17.10' as ILM is enabled.
2023-05-30T10:36:18.894-0400 INFO [esclientleg] eslegclient/connection.go:105 elasticsearch url: http://192.168.2.18:9200
2023-05-30T10:36:18.901-0400 INFO [publisher] pipeline/module.go:113 Beat name: kali
2023-05-30T10:36:18.904-0400 INFO [esclientleg] eslegclient/connection.go:105 elasticsearch url: http://192.168.2.18:9200
2023-05-30T10:36:18.941-0400 ERROR [esclientleg] transport/logging.go:37 Error dialing dial tcp 192.168.2.18:9200: connect: connection refused {"network": "tcp", "address": "192.168.2.18:9200"}
2023-05-30T10:36:18.942-0400 ERROR [esclientleg] eslegclient/connection.go:232 error connecting to Elasticsearch at http://192.168.2.18:9200: Get "http://192.168.2.18:9200": dial tcp 192.168.2.18:9200: connect: connection refused
2023-05-30T10:36:18.943-0400 ERROR instance/beat.go:1027 Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at http://192.168.2.18:9200: Get "http://192.168.2.18:9200": dial tcp 192.168.2.18:9200: connect: connection refused]
Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at http://192.168.2.18:9200: Get "http://192.168.2.18:9200": dial tcp 192.168.2.18:9200: connect: connection refused]
From our Windows Host Machine :
C:\Program Files\Winlogbeat>winlogbeat.exe setup -e
{"log.level":"info","@timestamp":"2023-05-30T10:43:21.136-0400","log.origin":{"file.name":"instance/beat.go","file.line":779},"message":"Home path: [C:\\Program Files\\Winlogbeat] Config path: [C:\\Program Files\\Winlogbeat] Data path: [C:\\Program Files\\Winlogbeat\\data] Logs path: [C:\\Program Files\\Winlogbeat\\logs]","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-05-30T10:43:21.141-0400","log.origin":{"file.name":"instance/beat.go","file.line":787},"message":"Beat ID: 8b031e24-391e-40d2-8773-be0064eae638","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2023-05-30T10:43:21.896-0400","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/provider_aws_ec2.go","file.line":81},"message":"read token request for getting IMDSv2 token returns empty: Put \"http://169.254.169.254/latest/api/token\": dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network.. No token in the metadata request will be used.","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-05-30T10:43:21.897-0400","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1299},"message":"Beat info","service.name":"winlogbeat","system_info":{"beat":{"path":{"config":"C:\\Program Files\\Winlogbeat","data":"C:\\Program Files\\Winlogbeat\\data","home":"C:\\Program Files\\Winlogbeat","logs":"C:\\Program Files\\Winlogbeat\\logs"},"type":"winlogbeat","uuid":"8b031e24-391e-40d2-8773-be0064eae638"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-05-30T10:43:21.898-0400","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1308},"message":"Build info","service.name":"winlogbeat","system_info":{"build":{"commit":"ae3e3f9194a937d20197a7be5d3cbbacaceeb9cc","libbeat":"8.8.0","time":"2023-05-23T01:36:11.000Z","version":"8.8.0"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-05-30T10:43:21.899-0400","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1311},"message":"Go runtime info","service.name":"winlogbeat","system_info":{"go":{"os":"windows","arch":"amd64","max_procs":4,"version":"go1.19.9"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-05-30T10:43:21.899-0400","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/add_cloud_metadata.go","file.line":100},"message":"add_cloud_metadata: hosting provider type not detected.","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-05-30T10:43:22.179-0400","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1317},"message":"Host info","service.name":"winlogbeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2023-05-27T03:24:47-04:00","name":"DESKTOP-66BME4Q","ip":["fe80::565e:d1c3:4ee6:9dd0","fe80::3928:2a82:86cc:a3a","169.254.89.133","fe80::8fd4:5421:717a:acc9","100.120.250.127","fe80::fc6f:4143:1b38:4230","192.168.204.1","fe80::1a2f:143:1ebb:c89b","192.168.56.1","fe80::a393:2cd0:6b57:431c","169.254.238.169","fe80::1ba2:75db:9b0c:e934","169.254.132.80","fe80::5e8f:71f7:3d0:9917","192.168.2.11","fe80::2af8:1d8f:1e9e:f593","169.254.98.231","::1","127.0.0.1"],"kernel_version":"10.0.19041.2965 (WinBuild.160101.0800)","mac":["00:05:9a:3c:7a:00","48:ba:4e:af:d3:3e","0a:00:27:00:00:0a","0a:00:27:00:00:07","f4:96:34:ee:4f:b3","f6:96:34:ee:4f:b2","f4:96:34:ee:4f:b2","f4:96:34:ee:4f:b6"],"os":{"type":"windows","family":"windows","platform":"windows","name":"Windows 10 Home","version":"10.0","major":10,"minor":0,"patch":0,"build":"19045.2965"},"timezone":"EDT","timezone_offset_sec":-14400,"id":"65a3e0e1-d18d-4baf-a4f8-4beb1d0b9b21"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-05-30T10:43:22.179-0400","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1346},"message":"Process info","service.name":"winlogbeat","system_info":{"process":{"cwd":"C:\\Program Files\\Winlogbeat","exe":"C:\\Program Files\\Winlogbeat\\winlogbeat.exe","name":"winlogbeat.exe","pid":20380,"ppid":4932,"start_time":"2023-05-30T10:42:59.673-0400"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-05-30T10:43:22.182-0400","log.origin":{"file.name":"instance/beat.go","file.line":330},"message":"Setup Beat: winlogbeat; Version: 8.8.0","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-05-30T10:43:24.870-0400","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":108},"message":"elasticsearch url: http://192.168.2.18:9200","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-05-30T10:43:24.871-0400","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":105},"message":"Beat name: DESKTOP-66BME4Q","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-05-30T10:43:24.872-0400","log.logger":"winlogbeat","log.origin":{"file.name":"beater/winlogbeat.go","file.line":70},"message":"State will be read from and persisted to C:\\Program Files\\Winlogbeat\\data\\.winlogbeat.yml","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-05-30T10:43:24.873-0400","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":108},"message":"elasticsearch url: http://192.168.2.18:9200","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-05-30T10:43:26.880-0400","log.logger":"esclientleg","log.origin":{"file.name":"transport/logging.go","file.line":38},"message":"Error dialing dial tcp 192.168.2.18:9200: connectex: No connection could be made because the target machine actively refused it.","service.name":"winlogbeat","network":"tcp","address":"192.168.2.18:9200","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-05-30T10:43:26.880-0400","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":235},"message":"error connecting to Elasticsearch at http://192.168.2.18:9200: Get \"http://192.168.2.18:9200\": dial tcp 192.168.2.18:9200: connectex: No connection could be made because the target machine actively refused it.","service.name":"winlogbeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-05-30T10:43:26.882-0400","log.origin":{"file.name":"instance/beat.go","file.line":1274},"message":"Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at http://192.168.2.18:9200: Get \"http://192.168.2.18:9200\": dial tcp 192.168.2.18:9200: connectex: No connection could be made because the target machine actively refused it.]","service.name":"winlogbeat","ecs.version":"1.6.0"}
Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at http://192.168.2.18:9200: Get "http://192.168.2.18:9200": dial tcp 192.168.2.18:9200: connectex: No connection could be made because the target machine actively refused it.]
Here is my elasticsearch.yml , check if there is any correction to be made, everything worked fine prior to integrateing the HTTP.SSL, but since implemented now nothing works :
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: 192.168.2.18
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200`
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["192.168.2.18"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
# ---------------------------------- Security ----------------------------------
#
# *** WARNING ***
#
# Elasticsearch security features are not enabled by default.
# These features are free, but require configuration changes to enable them.
# This means that users don’t have to provide credentials and can get full access
# to the cluster. Network connections are also not encrypted.
#
# To protect your data, we strongly encourage you to enable the Elasticsearch security features.
# Refer to the following documentation for instructions.
#
# https://www.elastic.co/guide/en/elasticsearch/reference/7.16/configuring-stack-security.html
# Enable security features
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
enabled: true
key: certs/elastic/elastic.key
certificate: certs/elastic/elastic.crt
certificate_authorities: certs/ca/ca.crt
# Enable encryption and mutual authentication between cluster nodes
#xpack.security.transport.ssl:
# enabled: true
#verification_mode: certificate
#keystore.path: certs/transport.p12
#truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
# cluster.initial_master_nodes: ["elastic"]
# Allow HTTP API connections from anywhere
# Connections are encrytpted and require user authentication
http.host: 0.0.0.0
