SIEM Zeek log data getting Error decoding JSON

Elastic Security SIEM
1

I'm trying to ingest Zeek network log data (formerly Bro) into SIEM (Beta 7.2). Finally got it configured to ingest the data but I'm getting tons of errors decoding JSON. Here's a sample:
ERROR#011readjson/json.go:52#011Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
AND
Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
I'm using filebeat to ingest directly into elasticsearch. Here's my filebeat.yml:

#============================== Kibana =====================================
setup.kibana:
host: "mykibana host info here"
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:

Array of hosts to connect to.

hosts: "my elasticsearch host info here"

#============================= Filebeat modules ===============================

filebeat.config.modules:

Glob pattern for configuration loading

path: ${path.config}/modules.d/*.yml

Here is my Zeek.yml:

  • module: zeek

    All logs

    connection:
    enabled: true
    var.paths:
    - "/usr/local/bro/spool/logger/conn.log"
    dns:
    enabled: true
    var.paths:
    - "/usr/local/bro/spool/logger/dns.log"
    http:
    enabled: true
    var.paths:
    - "/usr/local/bro/spool/logger/http.log"
    files:
    enabled: true
    var.paths:
    - "/usr/local/bro/spool/logger/files.log"
    ssl:
    enabled: true
    var.paths:
    - "/usr/local/bro/spool/logger/ssl.log"
    notice:
    enabled: true
    var.paths:
    - "/usr/local/bro/spool/logger/notice.log"
2

Is your Zeek configured to output logs in JSON format? Because it seems that the default is plain-text logs, and SIEM requires the logs in JSON.

According to the Zeek docs:

Some writer options are global (i.e., they affect all log filters using that log writer). For example, to change the output format of all ASCII logs to JSON format:

redef LogAscii::use_json = T;

3

Adrian, Thanks.
I found and applied this bro (zeek) script to change logs to JSON format and ISO date format:
(this assumes compiled (bin) Bro install directory structure, ie /opt/bro. So you may need to adjust for your install if source install; /usr/local/bro)

Configure bro to write JSON logs

mkdir -p /opt/bro/share/bro/site/scripts

sudo tee /opt/bro/share/bro/site/scripts/json-logs.bro << EOF

@load tuning/json-logs

redef LogAscii::json_timestamps = JSON::TS_ISO8601;

redef LogAscii::use_json = T;

EOF

sudo tee -a /opt/bro/share/bro/site/local.bro << EOF

Load policy for JSON output

@load scripts/json-logs

EOF

4

Thanks for updating with the solution!

I'll create an issue to make this clearer in our docs, maybe point to this gist.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.