I am using Zeek to analyze the pcaps in the offline mode. To run the pcap i use command "sudo zeek -r /path/to/file.pcap. It generates the logs (dns.log, http.log, ssl.log etc) in the /opt/zeek/logs/current folder. These logs are not in JSON format. In the inline mode with the help of this code redef LogAscii::use_json = T; zeek converts those logs into json which are then ready to be parsed to elasticsearch, which works fine for me. But how to convert those generated logs during offline execution to JSON so that they can be parsed to elasticsearch. I tried to use a logstash filter for conversion but of no use.
So here is how i run:
- make sure that Kibana and Elasticsearch services are running in the background.
- if i have to ingest logs to logstash then i enable "use logstash" in filebeat.yml otherwise it is disabled.
- then run the filebeat using filebeat -e command
- then finally run zeek -r /path/to/file.pcap
After that i get this error "ERROR [reader_json] readjson/json.go:57 Error decoding JSON: invalid character
#looking for beginning of value" and the reason for this is that the log files format starts with
#separator \x09 #set_separator , #empty_field (empty) #unset_field - #path dns #open 2020-11-06-10-18-29 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto trans_id rtt query qclass qclass_name qtype qtype_name rcode rcode_name AA TC RD RA Z answers TTLs rejected #types time string addr port addr port enum count interval string count string count string count string bool bool bool bool count vector[string] vector[interval] bool 1601918839.155711 C5yYobSodHUVqnxP6 192.168.1.10 50158 192.168.1.1 53 udp 23469 0.019095 edge.microsoft.com 1 C_INTERNET 1 A 0 NOERROR F F T T 0 edge-microsoft-com.a-0016.a-msedge.net,a-0016.a-msedge.net,184.108.40.206 2286.000000,46.000000,46.000000 F.
I would be happy to have some guidance to resolve this issue. Thank You.