JSON Parse error: Illegal unquoted character

[geetika_gopi]

Hello,

I am using rsyslog to send my auditd logs to logstash. Before reaching logstash, an rsyslog template is applied for better formatting.

I run into an error here

jsonlines - JSON parse error, original data now in message field {:message=>"Illegal unquoted character ((CTRL-CHAR, code 29)): has to be escaped using backslash to be included in string value\n at

Here is my raw log:

 "message" => "{\"@timestamp\":\"2021-10-16T10:08:51.072022-04:00\",\"@version\":\"1\",\"message\":\"type=SERVICE_START msg=audit(abc): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=NetworkManager comm=\\\"systemd\\\" exe=\\\"/usr/lib/systemd/systemd\\\" hostname=? addr=? terminal=? res=success'\u001DUID=\\\"root\\\" AUID=\\\"unset\\\"\",\"host\":\"\",\"host_ip\":\"abc\",\"vendor\":\"abc\",\"logsource\":\"\",\"severity_label\":\"info\",\"severity\":\"6\",\"facility_label\":\"local6\",\"facility\":\"22\",\"program\":\"tag_audit_log\",\"pid\":\"-\",\"rawmsg\":\"type=SERVICE_START msg=audit(abc): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=NetworkManager comm=\\\"systemd\\\" exe=\\\"/usr/lib/systemd/systemd\\\" hostname=? addr=? terminal=? res=success'\u001DUID=\\\"root\\\" AUID=\\\"unset\\\"\",\"syslogtag\":\"tag_audit_log:\"}"

Can someone help me out? I know it has to do with escaping some special character, but im not sure what it or how to do it.

Thank you

[Badger]

The error message suggests you are using a json_lines codec. If your data is not valid JSON then you will need to use a json filter. You could change the codec to plain and unconditionally pass the message through a json filter, or you could make the filter conditional. There is no way to specify UTF-8 characters directly in a logstash configuration, you will need to use a ruby filter. Rather than escaping the group separator character I just delete it....

if "_jsonparsefailure" in [tags] {
    ruby { code => 'event.set("message", event.get("message").gsub("\u001D".encode("utf-8"),  ""))' }
    json { source => "message" }

}