Can a scheduler be setup for filebeat to push logs only at midnight

[Kamal_Raj]

Wanted to try ES on one of our servers, where the load/CPU utilization is already high. Hence, can I schedule Filebeat to push logs only at midnight

I am aware that scan_frequency:86400 setting in Filebeat config, will push in an interval of 24 hours. But is there a provision, to mention time, i.e 12 midnight

@steffens, Could you please help

[Kamal_Raj]

Hi Team
Could anyone please help me on this. Im really stuck

[jsoriano (Jaime Soriano)]

Hi @Kamal_Raj,

filebeat and log pushing shouldn't cause a big overhead and in general it is a good idea to send the logs to a centralized server as soon as possible.

If in any case you need to schedule log forwarding at an specific time, in principlefilebeat doesn't provide a solution for that, but one thing you can try is to use cron for scheduling filebeat run --once at midnight. The --once flag makes filebeat to run only until all pending log files have been read. Take into account that processing a whole day of logs at once can also provoke a big peak in load depending on the volume of logs your application generates.