Send Logstash JSON to python

[neslog]

I'm trying to take the JSON output form Logstash and read it in with Python. Specifically I have the following configuration.

WinLogBeat --> JSON --> Logstash Beats receiver --> output to file

That all works well. When I try to parse the JSON output with python I keep getting Extra data errors. I've been searching around and everything looks like it's more focused on sending logs to logstash from python.

Please link me to any work already done. Any specific trips on getting Logstash JSON output parsed by python?

Thanks in advance for the help!

[yaauie (Ry Biesemeyer)]

What does your pipeline's output configuration look like? What codec is being used?

I presume that you're using the json_lines codec, which outputs the events as a series of newline-delimited JSON blobs; this format also called jsonlines or ndjson (Newline-Delimtied JSON).

If this is the case, when you're consuming in your python application, you'll need to split the input on the newline character before passing each chunk to a JSON parser. There is a python plugin that will do this for you called jsonlines.

[neslog]

yaauie - that was it! Thanks.