Logstash log4j2.properties configuration for logs rotation

[Kshema]

In our application, Logstash plain log file - logstas-plain.log is getting converted to a zip file logstash-plain-<date>.log.gz each day. The zip files are getting piled up and causing a memory issue.

Any help on configuring log4j2.properties to setup the logs rotation so that older zip files can get deleted either with SizeBasedTriggeringPolicy or TimeBasedTriggeringPolicy would be appreciable.

Thanks.

[AquaX (Andreas Helmer)]

In the log4j2.properties file you need to edit these settings:

appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling.policies.size.size = 100MB
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.max = 30

This means that the files will rollover at 100MB and 30 files will be kept.

Change those numbers to reflect what would work best for your environment and then restart logstash.

[Kshema]

Thank you, @AquaX.

The default max configuration was by default set to 30 in log4j2.properties but I can see around 50 logs in our logs path as the time based roll over approach is preceding over size based approach ( log file size not reaching 100MB before 1 day finishes).

appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval =1
appender.rolling.strategy.max = 30

I have reduced values for size for testing and seems like max file number works for size based roll over. Below are the configurations:

appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling.policies.size.size = 5KB
appender.rolling.strategy.max = 2

Can you please suggest on how appender.rolling.strategy.max value can be used for TimeBasedTriggeringPolicy as well.

If that's not a possible case, is there any other way to configure the deletion of old logs for time based roll over approach.

Thanks again

[AquaX (Andreas Helmer)]

The appender.rolling.strategy.max applies to both the TimeBasedTriggeringPolicy and to the SizeBasedTriggeringPolicy since they are both using the DefaultRolloverStrategy.

If there were 50 logs in your log path were those all of the same log type? The appender.rolling.strategy.max is the maximum for each log and not the maximum for the entire log folder.