Unable to delete ES logs older than 7 Days with Elastic version 6.x


(vijayakrishna Gade) #1

Hi,

I have configured Elasticsearch cluster with 4 nodes and i see everyday it's creating new cluster logging file in /va/log/elasticsearch

-rw-r--r--. 1 elasticsearch elasticsearch 5323 Nov 12 17:22 myescluster-2018-11-09-1.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 310862 Nov 13 03:17 myescluster-2018-11-12-1.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 67109443 Nov 14 08:23 gc.log.0
-rw-r--r--. 1 elasticsearch elasticsearch 443580 Nov 14 19:30 myescluster-2018-11-13-1.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 71803 Nov 15 00:33 myescluster-2018-11-14-1.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 67108990 Nov 15 17:29 gc.log.1
-rw-r--r--. 1 elasticsearch elasticsearch 1171038 Nov 15 18:01 myescluster-2018-11-15-1.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 1122541 Nov 15 19:41 myescluster-2018-11-15-2.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 705541 Nov 16 00:00 myescluster-2018-11-15-3.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 1144169 Nov 16 00:46 myescluster-2018-11-16-1.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 1086573 Nov 16 17:16 myescluster-2018-11-16-2.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 1081494 Nov 16 18:39 myescluster-2018-11-16-3.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 1069709 Nov 16 18:48 myescluster-2018-11-16-4.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 1118036 Nov 16 23:01 myescluster-2018-11-16-5.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 67109012 Nov 17 01:40 gc.log.2
-rw-r--r--. 1 elasticsearch elasticsearch 67109484 Nov 18 16:46 gc.log.3
-rw-r--r--. 1 elasticsearch elasticsearch 448297 Nov 19 19:52 myescluster-2018-11-16-6.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 11735 Nov 20 00:46 myescluster-2018-11-19-1.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 1074127 Nov 20 09:56 myescluster-2018-11-20-1.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 1075036 Nov 20 09:59 myescluster-2018-11-20-2.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch 1100643 Nov 20 10:13 myescluster-2018-11-20-3.log.gz

i have updated log4j2.properties file with below information to delete es cluster logs older than 7 days.

appender.rolling.type = RollingFile
appender.rolling.name = rolling
appender.rolling.fileName = {sys:es.logs.base_path}{sys:file.separator}{sys:es.logs.cluster_name}.log_ _appender.rolling.layout.type = PatternLayout_ _appender.rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] %marker%.-10000m%n_ _appender.rolling.filePattern = {sys:es.logs.base_path}{sys:file.separator}{sys:es.logs.cluster_name}-%d{yy$
appender.rolling.policies.type = Policies
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 1
appender.rolling.policies.time.modulate = true
appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling.policies.size.size = 128MB
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.fileIndex = nomax
appender.rolling.strategy.action.type = Delete
appender.rolling.strategy.action.basepath = {sys:es.logs.base_path}_ _appender.rolling.strategy.action.condition.type = IfFileName_ _appender.rolling.strategy.action.condition.glob = {sys:es.logs.cluster_name}-*
#appender.rolling.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
#appender.rolling.strategy.action.condition.nested_condition.exceeds = 2GB

appender.rolling.strategy.action.condition.nested_condition.type = IfLastModified
_appender.rolling.strategy.action.condition.nested_condition.age = 7D

after updating log properties file i have restarted elastiserch service on cluster, still i see old logs in the /var/log/elasticsearch, when these logs will be deleted and is there any other setting do i need to do for it.

I want to delete both logs older than 7 days.

  1. myescluster-yyyy-mm-dd-1.log.gz
  2. gc.log.x