Error decoding JSON: invalid character '\x00' looking for beginning of value

Elastic Stack Beats
1

Hello,

I am new to elastic filebeat and need some assistance with an error. We wrote a powershell script to pull down Office 365 audit logs and write them out to CSV files. These records also include a field that contains all of the event details in json format, which I am writing out to a separate json file. I am trying to use filebeat to ingest those json files to our SIEM server (Humio) but am getting the error below in /logs/filebeat.

Any help is appreciated.

Thanks.

/logs/filebeat:

|2019-11-26T12:05:49.956-0500|INFO|log/harvester.go:251|Harvester started for file: D:\scripts\Office365AuditLogs\json\AuditRecords-20191126-1220.json|
|---|---|---|---|
|2019-11-26T12:05:49.956-0500|ERROR|readjson/json.go:52|Error decoding JSON: invalid character '\x00' looking for beginning of value|
|2019-11-26T12:05:49.956-0500|ERROR|readjson/json.go:52|Error decoding JSON: invalid character '\x00' looking for beginning of value|

This is the contents of the AuditRecords-20191126-1220.json file (this particular file only has only two lines but the above error is happening for all the files):

{"CreationTime":"2019-11-26T12:02:14","Id":"a9575928-0591-4336-eb55-08d7726879d6","Operation":"FileAccessed","OrganizationId":"[org_id]","RecordType":6,"UserKey":"[userkey]@live.com","UserType":0,"Version":1,"Workload":"OneDrive","ClientIP":"1.1.1.100","ObjectId":"https:my.sharepoint.com\/[url].docx","UserId":"user@email.com","CorrelationId":"correlation_id","EventSource":"SharePoint","ItemType":"File","ListId":"[list_id]","ListItemUniqueId":"[listitemuniqueid]","Site":"[site]","UserAgent":"Microsoft Office Word\/16.0.11328.20468 (Windows\/10.0; Desktop WOW64; en-US; Desktop app;)","WebId":"[web_id]","SourceFileExtension":"docx","SiteUrl":"https:\/\/my.sharepoint.com\/url\/","SourceFileName":"[filename].docx","SourceRelativeUrl":"[url]"}
{"CreationTime":"2019-11-26T12:02:48","Id":"40871d2b-3077-483e-5b34-08d772688e78","Operation":"SearchQueryPerformed","OrganizationId":"[org_id]","RecordType":4,"UserKey":"[userkey]@live.com","UserType":0,"Version":1,"Workload":"SharePoint","ClientIP":"3.3.3.103","ObjectId":"object_id","UserId":"#user@email.com","CorrelationId":"correlation_id","EventSource":"SharePoint","ItemType":"Web","UserAgent":"Mozilla\/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident\/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0E; .NET4.0C; InfoPath.3)","EventData":"<ClientType>OneDriveWeb<\/ClientType>"}

And my filebeat.yml:

    #======================== Filebeat inputs =============================
filebeat.inputs:
- type: log
  enabled: true
  paths:
   - D:\scripts\Office365AuditLogs\json\*.json
  encoding: plain
  json.keys_under_root: true
  fields:
    app_id: office365_audit

queue.mem:
  events: 8000
  flush.min_events: 1000
  flush.timeout: 1s

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["https://[fqdn]:8443/internalapi"] #  -> gets redirected to ["$BASEURL/api/v1/ingest/elastic-bulk"]
  ssl.verification_mode: none
  username: joe_user
  password: [api key]
  compression_level: 5
  bulk_max_size: 200
  worker: 1
2

I have tried to reproduce the issue with the logs you have provided, but Filebeat was able to parse the lines properly. Could you please find the exact logs which lead to an error?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.