Read JSON fields with filebeat

Hi all,

I'm trying to read the data fields of a JSON message from my anti virus with filebeat, but somehow it keeps showing up as a full message.

It doesn't filter/detect the process.name and hostname and PID either.

Here's a example message I get:

May 5 10:58:36 app ERAServer[880] {"event_type":"Threat_Event","ipv4":"192.168.178.221","hostname":"gaming","source_uuid":"5a0ad926-a7e5-4673-967b-21f351135c14","occured":"05-May-2019 10:57:50","severity":"Warning","threat_type":"test file","threat_name":"Eicar","scanner_id":"HTTP filter","scan_id":"virlog.dat","engine_version":"19304 (20190505)","object_type":"file","object_uri":"https://www.eicar.org/download/eicar.com.txt","action_taken":"connection terminated","threat_handled":true,"need_restart":false,"username":"GAMING\\leon","processname":"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe","circumstances":"Event occurred during an attempt to access the web.","hash":"3395856CE81F2B7382DEE72602F798B642F14140"}

My filebeat config looks like this:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/syslog
  json.keys_under_root: true
  json.add_error_key: true
  processors:
     - decode_json_fields:
         fields: ["ipv4", "hostname", "object_type", "object_uri", "action_taken", "thread_name", "username"]
         process_array: false
         max_depth: 1
         target: ""
         overwrite_keys: true

It do get an error:
Error decoding JSON: invalid character 'M' looking for beginning of value

Most likely because it starts reading JSON from the beginning of the line, which is a timestamp, not JSON. But I am clueless on how to filter that out.

Would love some help here. I've been working and searching for a solutions for 2 weeks now.

UPDATE:
Fixed it using logstash instead.

Never mind, fixed it using logstash instead.

1 Like

I was gunna say that or use the Ingest API in Elasticsearch.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.