Rotating ES Logs

Hey all,

My ES logs are growing quite large on a daily basis. I understand that ES uses log4j for log rotation. But unfortunately I don't know enough about it to rotate my logs as needed. Currently ES logs can grow up to 8GB and overwhelm the partition.

This is my log4j file definition in /etc/elasticsearch/logging.yml:

file: #type: dailyRollingFile type: file file: ${path.logs}/${cluster.name}.log datePattern: "'.'yyyy-MM-dd" layout: type: pattern conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %.10000m%n"

How can I alter this definition so that log files never grow past 1gb and are gzipped once it gets past that point. Also how do I set the def so that zipped logs older than 3 days old get deleted.

Here's my full logging.yml file in case that's a help:

`# you can override this using by setting a system property, for example -Des.logger.level=DEBUG
es.logger.level: INFO
rootLogger: ${es.logger.level}, console, file
logger:

log action execution errors for easier debugging

action: DEBUG

deprecation logging, turn to DEBUG to see them

deprecation: INFO, deprecation_log_file

reduce the logging for aws, too much is logged under the default INFO

com.amazonaws: WARN

aws will try to do some sketchy JMX stuff, but its not needed.

com.amazonaws.jmx.SdkMBeanRegistrySupport: ERROR
com.amazonaws.metrics.AwsSdkMetrics: ERROR

org.apache.http: INFO

gateway

#gateway: DEBUG
#index.gateway: DEBUG

peer shard recovery

#indices.recovery: DEBUG

discovery

#discovery: TRACE

index.search.slowlog: TRACE, index_search_slow_log_file
index.indexing.slowlog: TRACE, index_indexing_slow_log_file

additivity:
index.search.slowlog: false
index.indexing.slowlog: false
deprecation: false

appender:
console:
type: console
layout:
type: consolePattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

file:
#type: dailyRollingFile
type: file
file: ${path.logs}/${cluster.name}.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %.10000m%n"

Use the following log4j-extras RollingFileAppender to enable gzip compression of log files.

For more information see https://logging.apache.org/log4j/extras/apidocs/org/apache/log4j/rolling/RollingFileAppender.html

#file:
#type: extrasRollingFile
#file: ${path.logs}/${cluster.name}.log
#rollingPolicy: timeBased
#rollingPolicy.FileNamePattern: ${path.logs}/${cluster.name}.log.%d{yyyy-MM-dd}.gz
#layout:
#type: pattern
#conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

deprecation_log_file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}_deprecation.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

index_search_slow_log_file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}_index_search_slowlog.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

index_indexing_slow_log_file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}_index_indexing_slowlog.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"`

Lastly if anyone can point me to a good log4j tutorial so I can have a better understanding of how this works, I'd appreciate that as well.

Why don't you fix the problem that is causing these large logs instead?