[SOLVED] How to manage HUGE log messages sent to ELK stack over-the-wire?

I've been using ELK for centralized logging service, and rsyslog as my shipper. Rsyslog wraps the log events in a JSON format.

However, some of our application logs are HUGE, e.g., stack traces, and when Logstash receives them, the log is truncated, resulting in _jsonparsefailure. I've tried sending the logs either via UDP/UDP6 and TCP/TCP6 to no avail.

How have some of you manage to solve this? I'm sure I'm not the only one in this predicament.


For rsysog users, see How to ship JSON logs via Rsyslog, especially the $MaxMessageSize setting. This along with the buffer_size setting solved my issues where large log events were being truncated.

Thanks for sharing this solution!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.