Hi, I Installed Zeek on an Ubuntu 22 VM and would like to send logs to Elasticsearch/Kibana using Filebeat. I followed Zeek Logs Intergation Tutorial but it's not able to send the logs. These are on separate Ubuntu 22 VM's on VMware Workstation 17 Pro (17.5.0 build-22583795).
Note: I also tested Filebeat's Suricata module on anotehr Ubuntu 22 VM and its successfully sending logs.
Thanks!
- OS's: Ubuntu 22
- Filebeat Version: 8.12
- Zeek Version: 6.2.0-dev.481
Here is the configuration:
(Zeek) /etc/filebeat/filebeat.yml:
filebeat.inputs:
- type: filestream
id: my-filestream-id
enabled: true
paths:
- /var/log/*.log
- /usr/local/zeek/logs/current/*.log
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 1
setup.kibana:
host: "172.16.1.40:5601"
output.elasticsearch:
hosts: ["172.16.1.40:9200"]
preset: balanced
username: "elastic"
password: "changeme"
ssl.verification_mode: "none"
processors:
- add_host_metadata:
when.not.contains.tags: forwarded
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
(Zeek) /etc/filebeat/modules.d/zeek.yml
# Module: zeek
# Docs: https://www.elastic.co/guide/en/beats/filebeat/main/filebeat-module-zeek.html
- module: zeek
capture_loss:
enabled: true
var.paths: ["/usr/local/zeek/logs/current/capture_loss.log"]
connection:
enabled: true
var.paths: ["/usr/local/zeek/logs/current/conn.log"]
dce_rpc:
enabled: false
dhcp:
enabled: false
var.paths: ["/usr/local/zeek/logs/current/dhcp.log"]
dnp3:
enabled: true
var.paths: ["/usr/local/zeek/logs/current/dnp3.log"]
dns:
enabled: true
var.paths: ["/usr/local/zeek/logs/current/dns.log"]
dpd:
enabled: false
files:
enabled: true
var.paths: ["/usr/local/zeek/logs/current/files.log"]
ftp:
enabled: true
var.paths: ["/usr/local/zeek/logs/current/ftp.log"]
http:
enabled: true
var.paths: ["/usr/local/zeek/logs/current/http.log"]
intel:
enabled: true
var.paths: ["/usr/local/zeek/logs/current/intel.log"]
.
.
.
Log files:
/var/log/filebeat/filebeat-20240212-1.ndjson
{"log.level":"info","@timestamp":"2024-02-12T14:50:28.497+0100","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":811},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-02-12T14:50:28.509+0100","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":819},"message":"Beat ID: 177ae49f-f656-491b-a3c1-8b9ba324d582","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-02-12T14:50:28.514+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1337},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/etc/filebeat","data":"/var/lib/filebeat","home":"/usr/share/filebeat","logs":"/var/log/filebeat"},"type":"filebeat","uuid":"177ae49f-f656-491b-a3c1-8b9ba324d582"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-02-12T14:50:28.514+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1346},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"27c592782c25906c968a41f0a6d8b1955790c8c5","libbeat":"8.12.0","time":"2024-01-10T21:05:10.000Z","version":"8.12.0"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-02-12T14:50:28.514+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1349},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":2,"version":"go1.20.12"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-02-12T14:50:28.515+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1355},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2024-02-12T13:27:44+01:00","containerized":false,"name":"zeek","ip":["127.0.0.1","::1","172.16.1.25"],"kernel_version":"6.5.0-15-generic","mac":["00:0c:29:59:87:cf","00:0c:29:59:87:d9"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"22.04.3 LTS (Jammy Jellyfish)","major":22,"minor":4,"patch":3,"codename":"jammy"},"timezone":"CET","timezone_offset_sec":3600,"id":"82d77215b56c4e4f8c20badab23010f4"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-02-12T14:50:28.515+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1384},"message":"Process info","service.name":"filebeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","perfmon","bpf","checkpoint_restore"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","perfmon","bpf","checkpoint_restore"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","perfmon","bpf","checkpoint_restore"],"ambient":null},"cwd":"/var/log","exe":"/usr/share/filebeat/bin/filebeat","name":"filebeat","pid":36505,"ppid":34253,"seccomp":{"mode":"disabled","no_new_privs":false},"start_time":"2024-02-12T14:50:27.520+0100"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-02-12T14:50:28.515+0100","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).createBeater","file.name":"instance/beat.go","file.line":334},"message":"Setup Beat: filebeat; Version: 8.12.0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2024-02-12T14:50:28.517+0100","log.logger":"cfgwarn","log.origin":{"function":"github.com/elastic/elastic-agent-libs/transport/tlscommon.(*Config).Validate.func1","file.name":"tlscommon/config.go","file.line":101},"message":"DEPRECATED: Treating the CommonName field on X.509 certificates as a host name when no Subject Alternative Names are present is going to be removed. Please update your certificates if needed. Will be removed in version: 8.0.0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-02-12T14:50:28.518+0100","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.makeES","file.name":"elasticsearch/elasticsearch.go","file.line":63},"message":"Applying performance preset 'balanced': {\n \"bulk_max_size\": 1600,\n \"compression_level\": 1,\n \"idle_connection_timeout\": \"3s\",\n \"queue\": {\n \"mem\": {\n \"events\": 3200,\n \"flush\": {\n \"min_events\": 1600,\n \"timeout\": \"10s\"\n }\n }\n },\n \"worker\": 1\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2024-02-12T14:50:28.518+0100","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.makeES","file.name":"elasticsearch/elasticsearch.go","file.line":66},"message":"Performance preset 'balanced' overrides user setting for field 'bulk_max_size'","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-02-12T14:50:28.518+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.NewConnection","file.name":"eslegclient/connection.go","file.line":122},"message":"elasticsearch url: http://172.16.1.40:9200","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2024-02-12T14:50:28.518+0100","log.logger":"tls","log.origin":{"function":"github.com/elastic/elastic-agent-libs/transport/tlscommon.(*TLSConfig).ToConfig","file.name":"tlscommon/tls_config.go","file.line":107},"message":"SSL/TLS verifications disabled.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-02-12T14:50:28.518+0100","log.logger":"publisher","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/publisher/pipeline.LoadWithSettings","file.name":"pipeline/module.go","file.line":105},"message":"Beat name: Zeek","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-02-12T14:50:28.518+0100","log.logger":"modules","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.newModuleRegistry","file.name":"fileset/modules.go","file.line":135},"message":"Enabled modules/filesets: ","service.name":"filebeat","ecs.version":"1.6.0"}
/var/log/filebeat/filebeat-20240212-2.ndjson
{"log.level":"info","@timestamp":"2024-02-12T15:46:46.363+0100","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":811},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-02-12T15:46:46.364+0100","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":819},"message":"Beat ID: 177ae49f-f656-491b-a3c1-8b9ba324d582","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-02-12T15:46:46.368+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1337},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/etc/filebeat","data":"/var/lib/filebeat","home":"/usr/share/filebeat","logs":"/var/log/filebeat"},"type":"filebeat","uuid":"177ae49f-f656-491b-a3c1-8b9ba324d582"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-02-12T15:46:46.368+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1346},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"27c592782c25906c968a41f0a6d8b1955790c8c5","libbeat":"8.12.0","time":"2024-01-10T21:05:10.000Z","version":"8.12.0"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-02-12T15:46:46.368+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1349},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":2,"version":"go1.20.12"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-02-12T15:46:46.369+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1355},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2024-02-12T13:27:44+01:00","containerized":false,"name":"zeek","ip":["127.0.0.1","::1","172.16.1.25"],"kernel_version":"6.5.0-15-generic","mac":["00:0c:29:59:87:cf","00:0c:29:59:87:d9"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"22.04.3 LTS (Jammy Jellyfish)","major":22,"minor":4,"patch":3,"codename":"jammy"},"timezone":"CET","timezone_offset_sec":3600,"id":"82d77215b56c4e4f8c20badab23010f4"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-02-12T15:46:46.369+0100","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.logSystemInfo","file.name":"instance/beat.go","file.line":1384},"message":"Process info","service.name":"filebeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","perfmon","bpf","checkpoint_restore"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","perfmon","bpf","checkpoint_restore"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","perfmon","bpf","checkpoint_restore"],"ambient":null},"cwd":"/home/owen","exe":"/usr/share/filebeat/bin/filebeat","name":"filebeat","pid":56001,"ppid":55953,"seccomp":{"mode":"disabled","no_new_privs":false},"start_time":"2024-02-12T15:46:45.340+0100"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2024-02-12T15:46:46.369+0100","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).createBeater","file.name":"instance/beat.go","file.line":334},"message":"Setup Beat: filebeat; Version: 8.12.0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2024-02-12T15:46:46.373+0100","log.logger":"cfgwarn","log.origin":{"function":"github.com/elastic/elastic-agent-libs/transport/tlscommon.(*Config).Validate.func1","file.name":"tlscommon/config.go","file.line":101},"message":"DEPRECATED: Treating the CommonName field on X.509 certificates as a host name when no Subject Alternative Names are present is going to be removed. Please update your certificates if needed. Will be removed in version: 8.0.0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-02-12T15:46:46.373+0100","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.makeES","file.name":"elasticsearch/elasticsearch.go","file.line":63},"message":"Applying performance preset 'balanced': {\n \"bulk_max_size\": 1600,\n \"compression_level\": 1,\n \"idle_connection_timeout\": \"3s\",\n \"queue\": {\n \"mem\": {\n \"events\": 3200,\n \"flush\": {\n \"min_events\": 1600,\n \"timeout\": \"10s\"\n }\n }\n },\n \"worker\": 1\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2024-02-12T15:46:46.373+0100","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.makeES","file.name":"elasticsearch/elasticsearch.go","file.line":66},"message":"Performance preset 'balanced' overrides user setting for field 'bulk_max_size'","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-02-12T15:46:46.373+0100","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.NewConnection","file.name":"eslegclient/connection.go","file.line":122},"message":"elasticsearch url: http://172.16.1.40:9200","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2024-02-12T15:46:46.373+0100","log.logger":"tls","log.origin":{"function":"github.com/elastic/elastic-agent-libs/transport/tlscommon.(*TLSConfig).ToConfig","file.name":"tlscommon/tls_config.go","file.line":107},"message":"SSL/TLS verifications disabled.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-02-12T15:46:46.373+0100","log.logger":"publisher","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/publisher/pipeline.LoadWithSettings","file.name":"pipeline/module.go","file.line":105},"message":"Beat name: Zeek","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-02-12T15:46:46.373+0100","log.logger":"modules","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/fileset.newModuleRegistry","file.name":"fileset/modules.go","file.line":135},"message":"Enabled modules/filesets: ","service.name":"filebeat","ecs.version":"1.6.0"}
systemctl status filebeat.service
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
Loaded: loaded (/lib/systemd/system/filebeat.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2024-02-16 11:46:43 CET; 46min ago
Docs: https://www.elastic.co/beats/filebeat
Main PID: 1289 (filebeat)
Tasks: 8 (limit: 4554)
Memory: 150.9M
CPU: 9.262s
CGroup: /system.slice/filebeat.service
└─1289 /usr/share/filebeat/bin/filebeat --environment systemd -c /etc/filebeat/filebeat.yml --path.home /usr/share/filebeat --path.config /etc/filebeat --path.data /var/lib/filebeat --path.logs /var/log/filebeat
Fra 16 12:31:53 Zeek filebeat[1289]: {"log.level":"error","@timestamp":"2024-02-16T12:31:53.973+0100","log.logger":"reload","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cfgfile.(*RunnerList).Reload","file.name":"cfgfile/list.go","file.line":138},"message":"Error creating runner from config: could not create module registry for filesets: fileset zeek/weird_stats is configured but doesn't exist","service.name":"filebeat","ecs.version":"1.6.0"}
Fra 16 12:32:08 Zeek filebeat[1289]: {"log.level":"error","@timestamp":"2024-02-16T12:32:08.979+0100","log.logger":"reload","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cfgfile.(*RunnerList).Reload","file.name":"cfgfile/list.go","file.line":138},"message":"Error creating runner from config: could not create module registry for filesets: fileset zeek/known_certs is configured but doesn't exist","service.name":"filebeat","ecs.version":"1.6.0"}
Fra 16 12:32:21 Zeek filebeat[1289]: {"log.level":"info","@timestamp":"2024-02-16T12:32:21.864+0100","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).logSnapshot","file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"memory":{"mem":{"usage":{"bytes":158535680}}}},"cpu":{"system":{"ticks":4080,"time":{"ms":40}},"total":{"ticks":9110,"time":{"ms":120},"value":9110},"user":{"ticks":5030,"time":{"ms":80}}},"handles":{"limit":{"hard":524288,"soft":524288},"open":19},"info":{"ephemeral_id":"72a0d70f-904e-41cb-87ad-62694275c885","uptime":{"ms":2734340},"version":"8.12.0"},"memstats":{"gc_next":44085064,"memory_alloc":29985208,"memory_total":466849760,"rss":129236992},"runtime":{"goroutines":53}},"filebeat":{"events":{"active":6,"added":308,"done":306},"harvester":{"open_files":8,"running":8,"started":1}},"libbeat":{"config":{"module":{"running":0},"reloads":2,"scans":2},"output":{"events":{"acked":306,"active":0,"batches":3,"total":306},"read":{"bytes":700,"errors":3},"write":{"bytes":36718}},"pipeline":{"clients":8,"events":{"active":6,"published":308,"total":308},"queue":{"acked":306}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.18,"15":0.09,"5":0.1,"norm":{"1":0.09,"15":0.045,"5":0.05}}}},"ecs.version":"1.6.0"}}
Fra 16 12:32:24 Zeek filebeat[1289]: {"log.level":"error","@timestamp":"2024-02-16T12:32:24.001+0100","log.logger":"reload","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cfgfile.(*RunnerList).Reload","file.name":"cfgfile/list.go","file.line":138},"message":"Error creating runner from config: could not create module registry for filesets: fileset zeek/netcontrol_catch_release is configured but doesn't exist","service.name":"filebeat","ecs.version":"1.6.0"}
Fra 16 12:32:39 Zeek filebeat[1289]: {"log.level":"error","@timestamp":"2024-02-16T12:32:39.012+0100","log.logger":"reload","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cfgfile.(*RunnerList).Reload","file.name":"cfgfile/list.go","file.line":138},"message":"Error creating runner from config: could not create module registry for filesets: fileset zeek/loaded_scripts is configured but doesn't exist","service.name":"filebeat","ecs.version":"1.6.0"}
Fra 16 12:32:51 Zeek filebeat[1289]: {"log.level":"info","@timestamp":"2024-02-16T12:32:51.864+0100","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).logSnapshot","file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"memory":{"mem":{"usage":{"bytes":158539776}}}},"cpu":{"system":{"ticks":4120,"time":{"ms":40}},"total":{"ticks":9180,"time":{"ms":70},"value":9180},"user":{"ticks":5060,"time":{"ms":30}}},"handles":{"limit":{"hard":524288,"soft":524288},"open":20},"info":{"ephemeral_id":"72a0d70f-904e-41cb-87ad-62694275c885","uptime":{"ms":2764340},"version":"8.12.0"},"memstats":{"gc_next":44085064,"memory_alloc":34236016,"memory_total":471100568,"rss":129236992},"runtime":{"goroutines":56}},"filebeat":{"events":{"active":6,"added":174,"done":174},"harvester":{"open_files":9,"running":9,"started":1}},"libbeat":{"config":{"module":{"running":0},"reloads":2,"scans":2},"output":{"events":{"acked":174,"active":0,"batches":3,"total":174},"read":{"bytes":667,"errors":3},"write":{"bytes":27363}},"pipeline":{"clients":9,"events":{"active":6,"published":174,"total":174},"queue":{"acked":174}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.11,"15":0.09,"5":0.09,"norm":{"1":0.055,"15":0.045,"5":0.045}}}},"ecs.version":"1.6.0"}}
Fra 16 12:32:54 Zeek filebeat[1289]: {"log.level":"error","@timestamp":"2024-02-16T12:32:54.017+0100","log.logger":"reload","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cfgfile.(*RunnerList).Reload","file.name":"cfgfile/list.go","file.line":138},"message":"Error creating runner from config: could not create module registry for filesets: fileset zeek/netcontrol_catch_release is configured but doesn't exist","service.name":"filebeat","ecs.version":"1.6.0"}
Fra 16 12:33:09 Zeek filebeat[1289]: {"log.level":"error","@timestamp":"2024-02-16T12:33:09.020+0100","log.logger":"reload","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cfgfile.(*RunnerList).Reload","file.name":"cfgfile/list.go","file.line":138},"message":"Error creating runner from config: could not create module registry for filesets: fileset zeek/unknown_protocols is configured but doesn't exist","service.name":"filebeat","ecs.version":"1.6.0"}
Fra 16 12:33:21 Zeek filebeat[1289]: {"log.level":"info","@timestamp":"2024-02-16T12:33:21.865+0100","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).logSnapshot","file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"memory":{"mem":{"usage":{"bytes":158543872}}}},"cpu":{"system":{"ticks":4140,"time":{"ms":20}},"total":{"ticks":9240,"time":{"ms":60},"value":9240},"user":{"ticks":5100,"time":{"ms":40}}},"handles":{"limit":{"hard":524288,"soft":524288},"open":21},"info":{"ephemeral_id":"72a0d70f-904e-41cb-87ad-62694275c885","uptime":{"ms":2794341},"version":"8.12.0"},"memstats":{"gc_next":44085064,"memory_alloc":39353280,"memory_total":476217832,"rss":129236992},"runtime":{"goroutines":59}},"filebeat":{"events":{"active":6,"added":309,"done":309},"harvester":{"open_files":10,"running":10,"started":1}},"libbeat":{"config":{"module":{"running":0},"reloads":2,"scans":2},"output":{"events":{"acked":309,"active":0,"batches":3,"total":309},"read":{"bytes":704,"errors":3},"write":{"bytes":36825}},"pipeline":{"clients":10,"events":{"active":6,"published":309,"total":309},"queue":{"acked":309}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.2,"15":0.1,"5":0.11,"norm":{"1":0.1,"15":0.05,"5":0.055}}}},"ecs.version":"1.6.0"}}
Fra 16 12:33:24 Zeek filebeat[1289]: {"log.level":"error","@timestamp":"2024-02-16T12:33:24.027+0100","log.logger":"reload","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cfgfile.(*RunnerList).Reload","file.name":"cfgfile/list.go","file.line":138},"message":"Error creating runner from config: could not create module registry for filesets: fileset zeek/netcontrol_drop is configured but doesn't exist","service.name":"filebeat","ecs.version":"1.6.0"}
filebeat test config
Config OK
filebeat test output
elasticsearch: http://172.16.1.40:9200...
parse url... OK
connection...
parse host... OK
dns lookup... OK
addresses: 172.16.1.40
dial up... OK
TLS... WARN secure connection disabled
talk to server... OK
version: 8.12.0