"json_error": "Error decoding JSON: invalid character 'S' looking for beginning of value

[iukea (jordan)]

Hello Currently getting the error
json_error": "Error decoding JSON: invalid character 'S' looking for beginning of value

Full error

     "input_type": "log",
      "json_error": "Error decoding JSON: invalid character 'S' looking for beginning of value",
      "log": "Starting server on port 5000/udp",
      "offset": 1106,
      "source": "/data/ciscoasa/log/ciscoasa.log",
      "type": "CiscoASA_Threats"

File beat YML is

paths:
- /data/ciscoasa/log/ciscoasa.log
input_type: log
document_type: CiscoASA_Threats


json.keys_under_root: true
json.add_error_key: true
json.message_key: log

Example of log file in /data/ciscoasa/log/ciscoasa.log

Starting server on port 5000/udp
Using selector: EpollSelector
Starting server on port 8443/tcp, use <Ctrl-C> to stop
{'timestamp': '2019-07-04T21:26:48.784247', 'src_ip': '37.1.205.153', 'payload_printable': '"GET / HTTP/1.1" 200 -'}
{'timestamp': '2019-07-04T21:26:51.789395', 'src_ip': '37.1.205.153', 'payload_printable': 'Request timed out: timeout('The read operation timed out',)'}
{'timestamp': '2019-07-04T21:29:23.410621', 'src_ip': '37.1.205.153', 'payload_printable': '"GET / HTTP/1.1" 200 -'}
{'timestamp': '2019-07-04T21:29:26.413366', 'src_ip': '37.1.205.153', 'payload_printable': 'Request timed out: timeout('The read operation timed out',)'}
{'timestamp': '2019-07-05T05:08:12.746232', 'src_ip': '178.128.194.144', 'payload_printable': '"GET / HTTP/1.1" 200 -'}
{'timestamp': '2019-07-05T05:08:15.749377', 'src_ip': '178.128.194.144', 'payload_printable': 'Request timed out: timeout('The read operation timed out',)'}
{'timestamp': '2019-07-05T07:14:55.251841', 'src_ip': '184.105.139.67', 'payload_printable': '"GET / HTTP/1.1" 200 -'}
Stopping server.
Starting server on port 5000/udp
Using selector: EpollSelector
Starting server on port 8443/tcp, use <Ctrl-C> to stop

Need some words of wisdom if any of you guys/ girls can help

[faec (Fae Charlton)]

Yes, the problem is that only some of your log lines are JSON -- in particular, "Starting server on port 5000/udp" is not valid JSON, and the JSON reader notices as soon as it tries to read the first character ('S').

A simple solution would be to add json.ignore_decoding_error: true to your configuration, but this would throw out all JSON errors, not just the ones caused by other types of output. A more robust alternative would be to filter out the lines that shouldn't use the JSON parser, e.g. in your filebeat.inputs configuration you could add:

include_lines: ['^\{']

however this version doesn't work for me as that regular expression is incorrect and I can't figure out why right now (I also tried '^{', '^\\{' and '^\\\{' XD), maybe someone else can jump in with the right syntax to match an open-brace at the start of a line