Out of memory issues with single log file

That could be enough memory, but if there's a lot of large lines and the events are all queued at once it could run into trouble. One thing to try is changing the queue size with e.g. queue.mem.events: 1024 (see the docs here) which would prevent too many lines from being loaded into memory at once.