Quoted json log message

[grants]

I have some event log files that are written in json, but they are wrapped in quotes.

Filebeats is dumping errors like this:
2017/08/01 18:37:12.103066 json.go:34: ERR Error decoding JSON: json: cannot unmarshal string into Go value of type map[string]interface {}

Input log example:
"{""System"":{""EventId"":""4634"",""Version"":""0"",""Channel"":""Security"",""ProviderName"":""Microsoft-Windows-Security-Auditing"",""Computer"":""hostname"",""EventRecordID"":""10204146"",""Keywords"":""Audit Success"",""Level"":""Log Always"",""Opcode"":""Info"",""Task"":""Logoff"",""ProcessID"":""848"",""ThreadID"":""27680"",""TimeCreated"":""1501611253559"",""UserId"":""""},""EventData"":{""TargetUserSid"":""dom\\\\hostb"",""TargetUserName"":""hostb"",""TargetDomainName"":""dob"",""TargetLogonId"":""0xed1a8f2"",""LogonType"":""3""}}"

Notice the multiple double quotes and the whole json message also wrapped in double quotes.

I need a way within filebeats to strip off the quotes or search and replace (sed) before the json processor. Or as an alternative, can I just ship the event like this to logstash and mutate the message field and then do a json extraction?

[tudor (Tudor Golubenco)]

I don't think we have a way to do this within Filebeat. I recommend sending it to LS and modifying it there.

[steffens (Steffen Siering)]

This is pretty much not valid json. Looks like you want to replace "" with " and remove the first/last ". Also replace \\\\ with \\. In Logstash one can try to do these replacements with the translate filter or the ruby filter and parse the resulting json with the json filter. Using Elasticsearch Ingest Node, you'd have to use painless, to script the replacements.

After doing the replacements, numbers and boolean values will still be presented as strings. This can be resolved with further post-processing in LS/ingest node or having a proper mapping template.

Is this some windows event log?

Is there a chance to adapt the log to contain valid json?