Log Deletion on Elasticsearch nodes based on lastModified

[mahesh_tangella (mahesh kumar tangella)]

Hello Team,

We are facing an issue with log deletion policy defined in log4j2.properties based on ifLastModified age. We want to delete all the log files which are larger than 2GB in size and older than 15 days.

So, we added the following properties to our log4j2.properties file on our existing Elasticsearch nodes.

appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.fileIndex = nomax
appender.rolling.strategy.action.type = Delete
appender.rolling.strategy.action.basepath = ${sys:es.logs.base_path}
appender.rolling.strategy.action.condition.type = IfFileName
appender.rolling.strategy.action.condition.glob = ${sys:es.logs.cluster_name}-*
appender.rolling.strategy.action.condition.nested_condition.type = IfAny
appender.rolling.strategy.action.condition.nested_condition.fileSize.type = IfAccumulatedFileSize
appender.rolling.strategy.action.condition.nested_condition.fileSize.exceeds = 2GB
appender.rolling.strategy.action.condition.nested_condition.lastMod.type = IfLastModified
appender.rolling.strategy.action.condition.nested_condition.lastMod.age = 15D

Upon restarting the Elasticsearch service on the nodes, all the log files which are larger than 2GB got deleted but the files older than 15Days don't get deleted. We even tried changing the lastModified condition to 2 days and waited for two days to check whether the files got deleted, but unfortunately no luck.

Can someone please suggest as to what's going wrong in the above configuration or any other alternatives we have, to delete log files after a certain limit is reached. This is crucial as we are running into disk space being full on Elasticsearch nodes.

Thanks,
Mahesh