How to test log rolling and deletion in elasticsearch with log4j2.properties?

Elastic Stack Elasticsearch
1

Hello,
We're using ES 6.1.1.
We want to delete logs which matching whatever of these two conditions: size and age.
For that, we have changed log4j2.properties file such as:

appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.fileIndex = nomax
appender.rolling.strategy.action.type = Delete
appender.rolling.strategy.action.basepath = ${sys:es.logs.base_path}
appender.rolling.strategy.action.condition.type = IfFileName
appender.rolling.strategy.action.condition.glob = ${sys:es.logs.cluster_name}-*
> appender.rolling.strategy.action.condition.nested_condition.type = IfAny
> appender.rolling.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
> appender.rolling.strategy.action.condition.nested_condition.exceeds = 2GB
> appender.rolling.strategy.action.condition.nested_condition.type = IfLastModified
> appender.rolling.strategy.action.condition.nested_condition.age = 7D

How could we test those changes?

Thanks and kind regards

2

I would just make the numbers smaller and run es with trace log so you can see if these are applied?

3

Good afternoon s1monw,

thanks for your reply but I must say I have logs older than 7days and grearter than 2Gb. So I have files accomplishing the needed conditions to be deleted, however, I haven't seen any change in last 10 days.

I can say I have changed logge debugger level with this command:

PUT /_cluster/settings
{"transient":{"logger._root":"TRACE"}}

and unexpectedly for me, I could see how logger deleted log files older than 7 days and greater then 2Gb but I don't know neither how nor why.

Later, I created new fake log files older than 7 days and greater then 2Gb I changed

PUT /_cluster/settings
{"transient":{"logger._root":"ERROR"}}

No change. I changed log level over to:

PUT /_cluster/settings
{"transient":{"logger._root":"TRACE"}}

but no change.

So that to say I don't understand how ES logger works and how test it to check the changes on log4j2.properties.

Any clue will be welcome.

Thanks and regards

4

$ sudo ls -laht /var/log/elasticsearch/*
...3.3G Jan 15 09:09 /var/log/elasticsearch/XTG_ElasticStatic-2018-01-13-1.log.gz
... 17K Jan 12 10:47 /var/log/elasticsearch/XTG_ElasticStatic.log
...4.6K Jan 12 10:47 /var/log/elasticsearch/XTG_ElasticStatic-2018-01-11-1.log.gz
...1.9K Jan 11 12:21 /var/log/elasticsearch/XTG_ElasticStatic-2018-01-10-1.log.gz
...3.1K Jan 10 10:07 /var/log/elasticsearch/XTG_ElasticStatic-2018-01-08-1.log.gz
...3.2K Jan 8 13:04 /var/log/elasticsearch/XTG_ElasticStatic-2017-12-22-1.log.gz
...3.3G Jan 1 09:05 /var/log/elasticsearch/XTG_ElasticStatic-2017-12-28-1.log.gz
... 350 Dec 25 09:11 /var/log/elasticsearch/XTG_ElasticStatic-2017-12-13-1.log.gz
... 0 Dec 22 12:24 /var/log/elasticsearch/XTG_ElasticStatic_index_indexing_slowlog.log
... 0 Dec 22 12:24 /var/log/elasticsearch/XTG_ElasticStatic_index_search_slowlog.log
... 0 Dec 22 12:24 /var/log/elasticsearch/XTG_ElasticStatic_deprecation.log

$ [HERE WE HAVE CHANGED LOGGER DEBUGGER LEVEL TO "TRACE" AND SUBSEQUENTLY WE GOT THIS OUTPUT]

$ sudo ls -laht /var/log/elasticsearch/*
... 95K Jan 23 17:11 /var/log/elasticsearch/XTG_ElasticStatic.log
...1.3K Jan 23 17:10 /var/log/elasticsearch/XTG_ElasticStatic-2018-01-12-1.log.gz
... 0 Dec 22 12:24 /var/log/elasticsearch/XTG_ElasticStatic_index_indexing_slowlog.log
... 0 Dec 22 12:24 /var/log/elasticsearch/XTG_ElasticStatic_index_search_slowlog.log
... 0 Dec 22 12:24 /var/log/elasticsearch/XTG_ElasticStatic_deprecation.log
[ ES LOGGER DELETED LOGS OLDER THAN 7DAYS AND GREATER 2GB BUT WE HAVE NOT BEEN ABLE TO REPRODUCE IT AGAIN]

5

This morning, I have caused a change on Logger Debugger Level to TRACE and I have seen the Delete Policy working again successfully but I don't know how to reproduce it without having to wait 24 hours to see it again.

I have tried to delete ${sys:es.logs.cluster_name}.log file but it didn't work.

Does anyone has any clue how could I test this log rolling and deletion with log4j2.properties?

Thanks

6

Good morning,

Does anyone know if it's possible to add two or more conditions on log4j2.properties?

Thanks and kind regards

7

We are not getting any answer about this issue
Would you advise me to ask this discussion in other forum?
Could you recommend which one?
Thanks and kind regards

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.