Filebeat CPU Load and Log File Size Issues

[chrispos (Chris)]

Hello everyone,

I'm relatively new to IT and currently facing some challenges with Filebeat that I hope the community can help me with.

The issue I'm encountering is related to the CPU load of Filebeat. The filebeat process takes around 120% cpu (2 cpu's) but this affect the performance of the server.
The log file I'm sending to the Logstash server gets renewed daily and, by the end of the day, it reaches a size of around 7 gigabytes. I've tried adjusting various settings, such as scan_frequency, but haven't had much luck in alleviating the CPU load.

To provide more context, the log file is a JSON that somstimes contains multiple lines in the message and section due to java stacktraces. I'm unsure if my Filebeat configuration is optimal for handling this type of log file, and any guidance on this matter would be greatly appreciated.
Here are some additional details about the server:
Memory: 7 gigabytes
CPUs: 2 (virtual)

Filebeat config

filebeat.inputs:
  - type: log
    enabled: true
    paths:
     - /opt/log.json
    multiline.pattern: '^{'
    multiline.negate: true
    multiline.match: after
    scan_frequency: 60s  # 20 seconden	
output.logstash:
........ rest of the config

Json object sample without a stacktrace, but with a multi-line message.

{
    "timestamp": "2023-11-15T15:39:52.317+01:00",
    "sequence": 27693701,
    "loggerClassName": "just a example",
    "loggerName": "i dont have any idea",
    "level": "DEBUG",
    "message": " SELECT  categorie, SUM(hoeveelheid * prijs_per_stuk) AS omzet 
     FROM   verkoop WHERE datum BETWEEN '2023-10-01
     ' AND '2023-10-31' GROUP BY  categorie ORDER BY  
     omzet DESC;",
    "threadName": "default task-482",",
    "threadName": "default task-922",
    "threadId": 2189,
    "mdc": {
    },
    "ndc": "",
    "hostName": "XXX",
    "processName": "XXX",
    "processId": 2129969
}

Json object with a multiline (stacktrace)

{
    "timestamp": "2023-11-15T15:39:52.317+01:00",
    "sequence": 27693701,
    "loggerClassName": "just a example",
    "loggerName": "i dont have any idea",
    "level": "DEBUG",
    "message": " Hello how are you ?",
    "threadName": "default task-922",
    "threadId": 2189,
    "mdc": {
    },
    "ndc": "",
    "hostName": "XXX",
    "processName": "XXX",
    "processId": 2129969
    "stackTrace": ": In the depths of the azure sea, Ariel, a curious mermaid with a voice as enchanting as the ocean's melody, dreamt of a world above. Against her 
	father's wishes, she traded her melodious voice for human legs, seeking love in the human realm. Amidst challenges and sacrifices, she discovered the true magic of 
	selfless love and found her place in both worlds, forever echoing in the sea's timeless tale."
	}
}

If anyone has experience with similar issues or has suggestions on how to optimize Filebeat for this scenario, please share your insights.

Thank you in advance