Hi there
On my company we have a server that has a huge amount of logs because of a lot of scheduled jobs and a few memory. So when filebeat turns up to collect em it takes all RAM to do so.
So the idea i had is to scan only the recent files on it. Then i put on filebeat.yml ignore_older: 15m and scan_frequency: 1m
Since that, the amount of logs collected by filebeat went from 12.000 to 0
It does not scan anything. And the logs are updated every 10 mins!
I changed the ignore_older: 2h same thing
The files to scan are huge so the filebeat.yml is configured like this for input type:
paths:
/opt/.../exploit/logs/job_*/ *.log
What did I missed? is the fact that are several folders and it only check folder modification?
Does that ignore_older option ignores new created log files?
