Logstash log4j2.properties logfiles are not deleting

kyle_che · April 4, 2022, 6:06pm

Looking at the log4j2.properties documentation it says to make the following changes to the log4j2.properties files. I've made them but also made changes for being logstash instead of elastic but I do not see any logfiles being deleted. any ideas?

-- elastic

appender.rolling.strategy.type = DefaultRolloverStrategy 
appender.rolling.strategy.action.type = Delete 
appender.rolling.strategy.action.basepath = ${sys:es.logs.base_path} 
appender.rolling.strategy.action.condition.type = IfFileName 
appender.rolling.strategy.action.condition.glob = ${sys:es.logs.cluster_name}-* 
appender.rolling.strategy.action.condition.nested_condition.type = IfLastModified 
appender.rolling.strategy.action.condition.nested_condition.age = 7D

-- logstash

appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.action.type = Delete
appender.rolling.strategy.action.basepath = ${sys:ls.logs.base_path}
appender.rolling.strategy.action.condition.type = IfFileName
appender.rolling.strategy.action.condition.glob = ${sys:ls.logs}/logstash-${sys:ls.log.format}-%d{yyyy-MM-dd}-%i.log.gz
appender.rolling.strategy.action.condition.nested_condition.type = IfLastModified
appender.rolling.strategy.action.condition.nested_condition.age = 30D

Badger · April 4, 2022, 6:41pm

This thread might help you.

kyle_che · April 4, 2022, 7:57pm

I tried what this article said but no luck ...

status = error
name = LogstashPropertiesConfig

appender.console.type = Console
appender.console.name = plain_console
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c]%notEmpty{[%X{pipeline.id}]} %m%n

appender.json_console.type = Console
appender.json_console.name = json_console
appender.json_console.layout.type = JSONLayout
appender.json_console.layout.compact = true
appender.json_console.layout.eventEol = true

appender.rolling.type = RollingFile
appender.rolling.name = plain_rolling
appender.rolling.fileName = ${sys:ls.logs}/logstash-${sys:ls.log.format}.log
appender.rolling.filePattern = ${sys:ls.logs}/logstash-${sys:ls.log.format}-%d{yyyy-MM-dd}-%i.log.gz
appender.rolling.policies.type = Policies
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 1
appender.rolling.policies.time.modulate = true
appender.rolling.layout.type = PatternLayout
appender.rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c]%notEmpty{[%X{pipeline.id}]} %m%n
appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling.policies.size.size = 100MB
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.max = 30

appender.rolling.strategy.delete.type = Delete
appender.rolling.strategy.delete.basepath = ${sys:ls.logs}
appender.rolling.strategy.delete.maxDepth = 1
appender.rolling.strategy.delete.condition.type = IfFileName
appender.rolling.strategy.delete.condition.nested_condition.type = IfLastModified
appender.rolling.strategy.delete.condition.nested_condition.age = 30D

appender.rolling.avoid_pipelined_filter.type = ScriptFilter
appender.rolling.avoid_pipelined_filter.script.type = Script
appender.rolling.avoid_pipelined_filter.script.name = filter_no_pipelined
appender.rolling.avoid_pipelined_filter.script.language = JavaScript
appender.rolling.avoid_pipelined_filter.script.value = ${sys:ls.pipeline.separate_logs} == false || !(logEvent.getContextData().containsKey("pipeline.id"))

appender.json_rolling.type = RollingFile
appender.json_rolling.name = json_rolling
appender.json_rolling.fileName = ${sys:ls.logs}/logstash-${sys:ls.log.format}.log
appender.json_rolling.filePattern = ${sys:ls.logs}/logstash-${sys:ls.log.format}-%d{yyyy-MM-dd}-%i.log.gz
appender.json_rolling.policies.type = Policies
appender.json_rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.json_rolling.policies.time.interval = 1
appender.json_rolling.policies.time.modulate = true
appender.json_rolling.layout.type = JSONLayout
appender.json_rolling.layout.compact = true
appender.json_rolling.layout.eventEol = true
appender.json_rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.json_rolling.policies.size.size = 100MB
appender.json_rolling.strategy.type = DefaultRolloverStrategy
appender.json_rolling.strategy.max = 30

appender.json_rolling.strategy.delete.type = Delete
appender.json_rolling.strategy.delete.basepath = ${sys:ls.logs}
appender.json_rolling.strategy.delete.maxDepth = 1
appender.json_rolling.strategy.delete.condition.type = IfFileName
appender.json_rolling.strategy.delete.condition.nested_condition.type = IfLastModified
appender.json_rolling.strategy.delete.condition.nested_condition.age = 30D

appender.json_rolling.avoid_pipelined_filter.type = ScriptFilter
appender.json_rolling.avoid_pipelined_filter.script.type = Script
appender.json_rolling.avoid_pipelined_filter.script.name = filter_no_pipelined
appender.json_rolling.avoid_pipelined_filter.script.language = JavaScript
appender.json_rolling.avoid_pipelined_filter.script.value = ${sys:ls.pipeline.separate_logs} == false || !(logEvent.getContextData().containsKey("pipeline.id"))

appender.routing.type = Routing
appender.routing.name = pipeline_routing_appender
appender.routing.routes.type = Routes
appender.routing.routes.script.type = Script
appender.routing.routes.script.name = routing_script
appender.routing.routes.script.language = JavaScript
appender.routing.routes.script.value = logEvent.getContextData().containsKey("pipeline.id") ? logEvent.getContextData().getValue("pipeline.id") : "sink";
appender.routing.routes.route_pipelines.type = Route
appender.routing.routes.route_pipelines.rolling.type = RollingFile
appender.routing.routes.route_pipelines.rolling.name = appender-${ctx:pipeline.id}
appender.routing.routes.route_pipelines.rolling.fileName = ${sys:ls.logs}/pipeline_${ctx:pipeline.id}.log
appender.routing.routes.route_pipelines.rolling.filePattern = ${sys:ls.logs}/pipeline_${ctx:pipeline.id}.%i.log.gz
appender.routing.routes.route_pipelines.rolling.layout.type = PatternLayout
appender.routing.routes.route_pipelines.rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n
appender.routing.routes.route_pipelines.rolling.policy.type = SizeBasedTriggeringPolicy
appender.routing.routes.route_pipelines.rolling.policy.size = 100MB
appender.routing.routes.route_pipelines.strategy.type = DefaultRolloverStrategy
appender.routing.routes.route_pipelines.strategy.max = 30
appender.routing.routes.route_sink.type = Route
appender.routing.routes.route_sink.key = sink
appender.routing.routes.route_sink.null.type = Null
appender.routing.routes.route_sink.null.name = drop-appender

rootLogger.level = ${sys:ls.log.level}
rootLogger.appenderRef.console.ref = ${sys:ls.log.format}_console
rootLogger.appenderRef.rolling.ref = ${sys:ls.log.format}_rolling
rootLogger.appenderRef.routing.ref = pipeline_routing_appender

# Slowlog

appender.console_slowlog.type = Console
appender.console_slowlog.name = plain_console_slowlog
appender.console_slowlog.layout.type = PatternLayout
appender.console_slowlog.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n

appender.json_console_slowlog.type = Console
appender.json_console_slowlog.name = json_console_slowlog
appender.json_console_slowlog.layout.type = JSONLayout
appender.json_console_slowlog.layout.compact = true
appender.json_console_slowlog.layout.eventEol = true

appender.rolling_slowlog.type = RollingFile
appender.rolling_slowlog.name = plain_rolling_slowlog
appender.rolling_slowlog.fileName = ${sys:ls.logs}/logstash-slowlog-${sys:ls.log.format}.log
appender.rolling_slowlog.filePattern = ${sys:ls.logs}/logstash-slowlog-${sys:ls.log.format}-%d{yyyy-MM-dd}-%i.log.gz
appender.rolling_slowlog.policies.type = Policies
appender.rolling_slowlog.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling_slowlog.policies.time.interval = 1
appender.rolling_slowlog.policies.time.modulate = true
appender.rolling_slowlog.layout.type = PatternLayout
appender.rolling_slowlog.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n
appender.rolling_slowlog.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling_slowlog.policies.size.size = 100MB
appender.rolling_slowlog.strategy.type = DefaultRolloverStrategy
appender.rolling_slowlog.strategy.max = 30

appender.json_rolling_slowlog.type = RollingFile
appender.json_rolling_slowlog.name = json_rolling_slowlog
appender.json_rolling_slowlog.fileName = ${sys:ls.logs}/logstash-slowlog-${sys:ls.log.format}.log
appender.json_rolling_slowlog.filePattern = ${sys:ls.logs}/logstash-slowlog-${sys:ls.log.format}-%d{yyyy-MM-dd}-%i.log.gz
appender.json_rolling_slowlog.policies.type = Policies
appender.json_rolling_slowlog.policies.time.type = TimeBasedTriggeringPolicy
appender.json_rolling_slowlog.policies.time.interval = 1
appender.json_rolling_slowlog.policies.time.modulate = true
appender.json_rolling_slowlog.layout.type = JSONLayout
appender.json_rolling_slowlog.layout.compact = true
appender.json_rolling_slowlog.layout.eventEol = true
appender.json_rolling_slowlog.policies.size.type = SizeBasedTriggeringPolicy
appender.json_rolling_slowlog.policies.size.size = 100MB
appender.json_rolling_slowlog.strategy.type = DefaultRolloverStrategy
appender.json_rolling_slowlog.strategy.max = 30

logger.slowlog.name = slowlog
logger.slowlog.level = trace
logger.slowlog.appenderRef.console_slowlog.ref = ${sys:ls.log.format}_console_slowlog
logger.slowlog.appenderRef.rolling_slowlog.ref = ${sys:ls.log.format}_rolling_slowlog
logger.slowlog.additivity = false

logger.licensereader.name = logstash.licensechecker.licensereader
logger.licensereader.level = error

kyle_che · April 4, 2022, 7:58pm

still showing stuff from last year ..

[Production] root@p-gp2-dbesdblogslogstash1-1.*****:/var/log/logstash
# ls -ltrh | wc -l
2197

[Production] root@p-gp2-dbesdblogslogstash1-1.*****:/var/log/logstash
# ls -ltrh | more
total 49G
-rwxr-xr-x 1 root     root      448 May 12  2020 check_grokparsefailure.sh*
-rw-r--r-- 1 root     root       26 May 12  2020 mail.email
-rw-r--r-- 1 logstash logstash    0 May 12  2020 logstash-slowlog-plain.log
-rw-r--r-- 1 logstash logstash 2.0M Jun 19  2021 logstash-plain-2021-06-18-2.log.gz
-rw-r--r-- 1 logstash logstash  22M Jun 19  2021 logstash-plain-2021-06-19-2.log.gz
-rw-r--r-- 1 logstash logstash  21M Jun 19  2021 logstash-plain-2021-06-19-3.log.gz
-rw-r--r-- 1 logstash logstash  22M Jun 19  2021 logstash-plain-2021-06-19-4.log.gz
-rw-r--r-- 1 logstash logstash  24M Jun 19  2021 logstash-plain-2021-06-19-5.log.gz
-rw-r--r-- 1 logstash logstash  24M Jun 19  2021 logstash-plain-2021-06-19-6.log.gz
-rw-r--r-- 1 logstash logstash  23M Jun 19  2021 logstash-plain-2021-06-19-7.log.gz
-rw-r--r-- 1 logstash logstash 7.1M Jun 20  2021 logstash-plain-2021-06-19-8.log.gz
-rw-r--r-- 1 logstash logstash  23M Jun 20  2021 logstash-plain-2021-06-20-2.log.gz
-rw-r--r-- 1 logstash logstash  20M Jun 20  2021 logstash-plain-2021-06-20-3.log.gz
-rw-r--r-- 1 logstash logstash  23M Jun 20  2021 logstash-plain-2021-06-20-4.log.gz
-rw-r--r-- 1 logstash logstash  24M Jun 20  2021 logstash-plain-2021-06-20-5.log.gz
-rw-r--r-- 1 logstash logstash  23M Jun 20  2021 logstash-plain-2021-06-20-6.log.gz
-rw-r--r-- 1 logstash logstash  22M Jun 20  2021 logstash-plain-2021-06-20-7.log.gz
-rw-r--r-- 1 logstash logstash 7.1M Jun 21  2021 logstash-plain-2021-06-20-8.log.gz
-rw-r--r-- 1 logstash logstash  22M Jun 21  2021 logstash-plain-2021-06-21-2.log.gz
-rw-r--r-- 1 logstash logstash  18M Jun 21  2021 logstash-plain-2021-06-21-3.log.gz
-rw-r--r-- 1 logstash logstash  19M Jun 21  2021 logstash-plain-2021-06-21-4.log.gz
-rw-r--r-- 1 logstash logstash  23M Jun 21  2021 logstash-plain-2021-06-21-5.log.gz
-rw-r--r-- 1 logstash logstash  23M Jun 21  2021 logstash-plain-2021-06-21-6.log.gz
-rw-r--r-- 1 logstash logstash  23M Jun 21  2021 logstash-plain-2021-06-21-7.log.gz
-rw-r--r-- 1 logstash logstash 6.5M Jun 22  2021 logstash-plain-2021-06-21-8.log.gz
-rw-r--r-- 1 logstash logstash  23M Jun 22  2021 logstash-plain-2021-06-22-2.log.gz
-rw-r--r-- 1 logstash logstash  21M Jun 22  2021 logstash-plain-2021-06-22-3.log.gz
-rw-r--r-- 1 logstash logstash  22M Jun 22  2021 logstash-plain-2021-06-22-4.log.gz
-rw-r--r-- 1 logstash logstash  24M Jun 22  2021 logstash-plain-2021-06-22-5.log.gz
-rw-r--r-- 1 logstash logstash  22M Jun 22  2021 logstash-plain-2021-06-22-6.log.gz
-rw-r--r-- 1 logstash logstash  23M Jun 22  2021 logstash-plain-2021-06-22-7.log.gz
-rw-r--r-- 1 logstash logstash 4.6M Jun 23  2021 logstash-plain-2021-06-22-8.log.gz
-rw-r--r-- 1 logstash logstash  22M Jun 23  2021 logstash-plain-2021-06-23-2.log.gz
-rw-r--r-- 1 logstash logstash  19M Jun 23  2021 logstash-plain-2021-06-23-3.log.gz
-rw-r--r-- 1 logstash logstash  23M Jun 23  2021 logstash-plain-2021-06-23-4.log.gz

Badger · April 4, 2022, 8:16pm

It does not look like that to me. The appender.rolling.strategy section in the linked article is

appender.rolling.strategy.max = 30
appender.rolling.strategy.delete.type = Delete
appender.rolling.strategy.delete.basePath = ${sys:ls.logs}
appender.rolling.strategy.delete.maxDepth = 1
appender.rolling.strategy.delete.ifLastModified.type = IfLastModified
appender.rolling.strategy.delete.ifLastModified.age = 30d

whereas yours is

appender.rolling.strategy.max = 30
appender.rolling.strategy.delete.type = Delete
appender.rolling.strategy.delete.basepath = ${sys:ls.logs}
appender.rolling.strategy.delete.maxDepth = 1
appender.rolling.strategy.delete.condition.type = IfFileName
appender.rolling.strategy.delete.condition.nested_condition.type = IfLastModified
appender.rolling.strategy.delete.condition.nested_condition.age = 30D

I do not understand log4j well enough to know whether they are equivalent.

Also, note that in the linked article I said that the size based condition for deletion was applied to appender.rolling based on the appender.json_rolling. I do not know whether other settings are applied to the wrong appender.

kyle_che · April 4, 2022, 8:53pm

i went and put exactly what you pasted above and still no luck.

status = error
name = LogstashPropertiesConfig

appender.console.type = Console
appender.console.name = plain_console
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c]%notEmpty{[%X{pipeline.id}]} %m%n

appender.json_console.type = Console
appender.json_console.name = json_console
appender.json_console.layout.type = JSONLayout
appender.json_console.layout.compact = true
appender.json_console.layout.eventEol = true

appender.rolling.type = RollingFile
appender.rolling.name = plain_rolling
appender.rolling.fileName = ${sys:ls.logs}/logstash-${sys:ls.log.format}.log
appender.rolling.filePattern = ${sys:ls.logs}/logstash-${sys:ls.log.format}-%d{yyyy-MM-dd}-%i.log.gz
appender.rolling.policies.type = Policies
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 1
appender.rolling.policies.time.modulate = true
appender.rolling.layout.type = PatternLayout
appender.rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c]%notEmpty{[%X{pipeline.id}]} %m%n
appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling.policies.size.size = 100MB
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.max = 30

appender.rolling.strategy.delete.type = Delete
appender.rolling.strategy.delete.basePath = ${sys:ls.logs}
appender.rolling.strategy.delete.maxDepth = 1
appender.rolling.strategy.delete.ifLastModified.type = IfLastModified
appender.rolling.strategy.delete.ifLastModified.age = 30d

appender.rolling.avoid_pipelined_filter.type = ScriptFilter
appender.rolling.avoid_pipelined_filter.script.type = Script
appender.rolling.avoid_pipelined_filter.script.name = filter_no_pipelined
appender.rolling.avoid_pipelined_filter.script.language = JavaScript
appender.rolling.avoid_pipelined_filter.script.value = ${sys:ls.pipeline.separate_logs} == false || !(logEvent.getContextData().containsKey("pipeline.id"))

appender.json_rolling.type = RollingFile
appender.json_rolling.name = json_rolling
appender.json_rolling.fileName = ${sys:ls.logs}/logstash-${sys:ls.log.format}.log
appender.json_rolling.filePattern = ${sys:ls.logs}/logstash-${sys:ls.log.format}-%d{yyyy-MM-dd}-%i.log.gz
appender.json_rolling.policies.type = Policies
appender.json_rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.json_rolling.policies.time.interval = 1
appender.json_rolling.policies.time.modulate = true
appender.json_rolling.layout.type = JSONLayout
appender.json_rolling.layout.compact = true
appender.json_rolling.layout.eventEol = true
appender.json_rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.json_rolling.policies.size.size = 100MB
appender.json_rolling.strategy.type = DefaultRolloverStrategy
appender.json_rolling.strategy.max = 30

appender.json_rolling.strategy.delete.type = Delete
appender.json_rolling.strategy.delete.basePath = ${sys:ls.logs}
appender.json_rolling.strategy.delete.maxDepth = 1
appender.json_rolling.strategy.delete.ifLastModified.type = IfLastModified
appender.json_rolling.strategy.delete.ifLastModified.age = 30d

appender.json_rolling.avoid_pipelined_filter.type = ScriptFilter
appender.json_rolling.avoid_pipelined_filter.script.type = Script
appender.json_rolling.avoid_pipelined_filter.script.name = filter_no_pipelined
appender.json_rolling.avoid_pipelined_filter.script.language = JavaScript
appender.json_rolling.avoid_pipelined_filter.script.value = ${sys:ls.pipeline.separate_logs} == false || !(logEvent.getContextData().containsKey("pipeline.id"))

appender.routing.type = Routing
appender.routing.name = pipeline_routing_appender
appender.routing.routes.type = Routes
appender.routing.routes.script.type = Script
appender.routing.routes.script.name = routing_script
appender.routing.routes.script.language = JavaScript
appender.routing.routes.script.value = logEvent.getContextData().containsKey("pipeline.id") ? logEvent.getContextData().getValue("pipeline.id") : "sink";
appender.routing.routes.route_pipelines.type = Route
appender.routing.routes.route_pipelines.rolling.type = RollingFile
appender.routing.routes.route_pipelines.rolling.name = appender-${ctx:pipeline.id}
appender.routing.routes.route_pipelines.rolling.fileName = ${sys:ls.logs}/pipeline_${ctx:pipeline.id}.log
appender.routing.routes.route_pipelines.rolling.filePattern = ${sys:ls.logs}/pipeline_${ctx:pipeline.id}.%i.log.gz
appender.routing.routes.route_pipelines.rolling.layout.type = PatternLayout
appender.routing.routes.route_pipelines.rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n
appender.routing.routes.route_pipelines.rolling.policy.type = SizeBasedTriggeringPolicy
appender.routing.routes.route_pipelines.rolling.policy.size = 100MB
appender.routing.routes.route_pipelines.strategy.type = DefaultRolloverStrategy
appender.routing.routes.route_pipelines.strategy.max = 30
appender.routing.routes.route_sink.type = Route
appender.routing.routes.route_sink.key = sink
appender.routing.routes.route_sink.null.type = Null
appender.routing.routes.route_sink.null.name = drop-appender

rootLogger.level = ${sys:ls.log.level}
rootLogger.appenderRef.console.ref = ${sys:ls.log.format}_console
rootLogger.appenderRef.rolling.ref = ${sys:ls.log.format}_rolling
rootLogger.appenderRef.routing.ref = pipeline_routing_appender

# Slowlog

appender.console_slowlog.type = Console
appender.console_slowlog.name = plain_console_slowlog
appender.console_slowlog.layout.type = PatternLayout
appender.console_slowlog.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n

appender.json_console_slowlog.type = Console
appender.json_console_slowlog.name = json_console_slowlog
appender.json_console_slowlog.layout.type = JSONLayout
appender.json_console_slowlog.layout.compact = true
appender.json_console_slowlog.layout.eventEol = true

appender.rolling_slowlog.type = RollingFile
appender.rolling_slowlog.name = plain_rolling_slowlog
appender.rolling_slowlog.fileName = ${sys:ls.logs}/logstash-slowlog-${sys:ls.log.format}.log
appender.rolling_slowlog.filePattern = ${sys:ls.logs}/logstash-slowlog-${sys:ls.log.format}-%d{yyyy-MM-dd}-%i.log.gz
appender.rolling_slowlog.policies.type = Policies
appender.rolling_slowlog.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling_slowlog.policies.time.interval = 1
appender.rolling_slowlog.policies.time.modulate = true
appender.rolling_slowlog.layout.type = PatternLayout
appender.rolling_slowlog.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n
appender.rolling_slowlog.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling_slowlog.policies.size.size = 100MB
appender.rolling_slowlog.strategy.type = DefaultRolloverStrategy
appender.rolling_slowlog.strategy.max = 30

appender.json_rolling_slowlog.type = RollingFile
appender.json_rolling_slowlog.name = json_rolling_slowlog
appender.json_rolling_slowlog.fileName = ${sys:ls.logs}/logstash-slowlog-${sys:ls.log.format}.log
appender.json_rolling_slowlog.filePattern = ${sys:ls.logs}/logstash-slowlog-${sys:ls.log.format}-%d{yyyy-MM-dd}-%i.log.gz
appender.json_rolling_slowlog.policies.type = Policies
appender.json_rolling_slowlog.policies.time.type = TimeBasedTriggeringPolicy
appender.json_rolling_slowlog.policies.time.interval = 1
appender.json_rolling_slowlog.policies.time.modulate = true
appender.json_rolling_slowlog.layout.type = JSONLayout
appender.json_rolling_slowlog.layout.compact = true
appender.json_rolling_slowlog.layout.eventEol = true
appender.json_rolling_slowlog.policies.size.type = SizeBasedTriggeringPolicy
appender.json_rolling_slowlog.policies.size.size = 100MB
appender.json_rolling_slowlog.strategy.type = DefaultRolloverStrategy
appender.json_rolling_slowlog.strategy.max = 30

logger.slowlog.name = slowlog
logger.slowlog.level = trace
logger.slowlog.appenderRef.console_slowlog.ref = ${sys:ls.log.format}_console_slowlog
logger.slowlog.appenderRef.rolling_slowlog.ref = ${sys:ls.log.format}_rolling_slowlog
logger.slowlog.additivity = false

logger.licensereader.name = logstash.licensechecker.licensereader
logger.licensereader.level = error

Badger · April 4, 2022, 9:10pm

What are the permissions on /var/log/logstash? Every time I let yum upgrade logstash it gets changed to root:root and 755, at which point I have to chgrp and chmod it to get old files deleted automatically.

Other than that, it is really a log4j question, not a logstash question, and I have reached the limit of my knowledge.

kyle_che · April 4, 2022, 10:42pm

seems that everything is owned by logstash

[Production] root@p-gp2-dbesdblogslogstash1-1.*****.com:/var/log/logstash
# ls -ltrh | more
total 40G
-rwxr-xr-x 1 root     root      448 May 12  2020 check_grokparsefailure.sh*
-rw-r--r-- 1 root     root       26 May 12  2020 mail.email
-rw-r--r-- 1 logstash logstash    0 May 12  2020 logstash-slowlog-plain.log
-rw-r--r-- 1 logstash logstash 2.0M Jun 19  2021 logstash-plain-2021-06-18-2.log.gz
-rw-r--r-- 1 logstash logstash  22M Jun 19  2021 logstash-plain-2021-06-19-2.log.gz
-rw-r--r-- 1 logstash logstash  21M Jun 19  2021 logstash-plain-2021-06-19-3.log.gz
-rw-r--r-- 1 logstash logstash  22M Jun 19  2021 logstash-plain-2021-06-19-4.log.gz
-rw-r--r-- 1 logstash logstash  24M Jun 19  2021 logstash-plain-2021-06-19-5.log.gz
-rw-r--r-- 1 logstash logstash  24M Jun 19  2021 logstash-plain-2021-06-19-6.log.gz

Badger · April 4, 2022, 10:45pm

What does ls -ld /var/log/logstash show?

kyle_che · April 5, 2022, 7:25pm

Production] root@p-gp2-dbesdblogslogstash1-1.*****.com:/root
# ls -ld /var/log/logstash
drwxr-xr-x 3 logstash logstash 163840 Apr  5 17:37 /var/log/logstash/

system · May 3, 2022, 7:25pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
How to test log rolling and deletion in elasticsearch with log4j2.properties? Elasticsearch	7	5690	March 15, 2018
Log4j2 questions Elasticsearch	1	592	November 22, 2017
Log Deletion on Elasticsearch nodes based on lastModified Elasticsearch	1	322	February 7, 2023
Log4j2.properties doubt Elasticsearch	2	186	November 1, 2022
Logstash log4j2.properties deleting all the compressed files Logstash	1	1245	December 31, 2021

Logstash log4j2.properties logfiles are not deleting

Related topics