Logstash logs filling up disk. How to configure log4j?

Hi all,

We are currently facing an issue where Logstash fills up disk space on some of the nodes on our K8s cluster by outputting entire events to stdout. I am aware that I can change the loglevel of Logstash as a whole or that of the Elasticsearch output plugin (the one generating the big outputs to stdout). Currently I can do:

logger.elasticsearchoutput.name = logstash.outputs.elasticsearch
logger.elasticsearchoutput.level = error

However, this would also silence other important warnings. Would it be possible to configure the log4j loggers to filter out specific warnings containing some text? I have been searching, but cannot find on how to do it in Logstash.

I know there is a failure_type_logging_whitelist option in the ES output plugin but that does not work: 400 and 404 errors do not honour the failure_type_logging_whitelist option · Issue #965 · logstash-plugins/logstash-output-elasticsearch · GitHub

Thanks in advance.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.