Filebeat + zeekmodule to elastic + kibana

[tmans1991]

hi

im using a filebeat + zeekmodule machine to send .log files (written in JSON) to
an elastic + kibana machine

i am sending conn.logg dns.log http.log files.log ssl.log and notice.log
but i am not sending :

capture_loss.log
ntp.log
smtp.log
stdout.log
kerberos.log
radius.log
snmp.log
syslog.log
dhcp.log
known_services.log
reporter.log
software.log
weird.log
mysql.log
sip.log
ssl.log
x509.log
dpd.log
smb_files.log
stats.log
ntlm.log
smb_mapping.log
stderr.log

can these be send and parsed with the zeek module aswell to the elastic + kibana machine?

[Kaiyan_Sheng]

Hi @tmans1991, thanks for posting your question here. The zeek module in Filebeat supports a lot of filesets, for example: capture_loss, coonection, dce_rpc, dhcp, dns and etc. If you only want to parse specific log type, you can do that by enable/disable filesets in zeek.yml:

- module: zeek
  capture_loss:
    enabled: false
  connection:
    enabled: true
  dce_rpc:
    enabled: false
  dhcp:
    enabled: false
  dnp3:
    enabled: false
  dns:
    enabled: true
  dpd:
    enabled: false
  files:
    enabled: true
  ftp:
    enabled: false
  http:
    enabled: true
  irc:
    enabled: false
  kerberos:
    enabled: false
  modbus:
    enabled: false
  mysql:
    enabled: false
  notice:
    enabled: true
  ntlm:
    enabled: false
  ocsp:
    enabled: false
  pe:
    enabled: false
  radius:
    enabled: false
  rdp:
    enabled: false
  rfb:
    enabled: false
  signatures:
    enabled: false
  sip:
    enabled: false
  smb_cmd:
    enabled: false
  smb_files:
    enabled: false
  smb_mapping:
    enabled: false
  smtp:
    enabled: false
  snmp:
    enabled: false
  socks:
    enabled: false
  ssh:
    enabled: false
  ssl:
    enabled: true
  stats:
    enabled: false
  syslog:
    enabled: false
  traceroute:
    enabled: false
  tunnel:
    enabled: false
  weird:
    enabled: false
  x509:
    enabled: false

[stcdarrell (Darrell Miller)]

this is what i needed, thank you

[stcdarrell (Darrell Miller)]

@Kaiyan_Sheng are you sure?
i just tried this..

and i get this error: Exiting: "Fileset zeek/dhcp is configured but doesn't exist"

    - module: zeek
  # All logs
  connection:
    enabled: true
    var.path: ["/var/log/zeek/conn.log"]
  dns:
    enabled: true
    var.path: ["/var/log/zeek/dns.log"]
  http:
    enabled: true
    var.path: ["/var/log/zeek/httpd.log"]
  files:
    enabled: true
    var.path: ["/var/log/zeek/files.log"]
  ssl:
    enabled: true
    var.path: ["/var/log/zeek/ssl.log"]
  notice:
    enabled: true
    var.path: ["/var/log/zeek/notice.log"]
  dhcp:
    enabled: true
    var.paths: ["/var/log/zeek/dhcp.log"]
  stats:
    enabled: true
    var.paths: ["/var/log/zeek/stats.log"]

  # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

[stcdarrell (Darrell Miller)]

@Kaiyan_Sheng

i just dug through the filebeat folder.. if you look in
/usr/share/filebeat/module/zeek

there is a folder for each "parser" (i guess thats the right word)
there is currently only a parser for :

• connection
• dns
• files
• http
• notice
• ssl

i dont think it would be hard to write for the other files, but unfortunately they arent there.. i looked in the 7.4.1 folder.. i'd love for you to prove me wrong, i could REALLY use the other logs in es.

Darrell

[Kaiyan_Sheng]

Yep you are correct. We already added support for some other files in https://github.com/elastic/beats/pull/14150. But this won't be released probably till 7.6 unfortunately. If there are other files you'd like we add support for, please feel free to open an issue in github!

[stcdarrell (Darrell Miller)]

thank you! i'm in no mad rush for those other files, but i am looking forward to them.
thanks for the clarification

Darrell