Hi all,
We are currently facing an issue where Logstash fills up disk space on some of the nodes on our K8s cluster by outputting entire events to stdout. I am aware that I can change the loglevel of Logstash as a whole or that of the Elasticsearch output plugin (the one generating the big outputs to stdout). Currently I can do:
logger.elasticsearchoutput.name = logstash.outputs.elasticsearch
logger.elasticsearchoutput.level = error
However, this would also silence other important warnings. Would it be possible to configure the log4j loggers to filter out specific warnings containing some text? I have been searching, but cannot find on how to do it in Logstash.
I know there is a failure_type_logging_whitelist option in the ES output plugin but that does not work: 400 and 404 errors do not honour the failure_type_logging_whitelist option · Issue #965 · logstash-plugins/logstash-output-elasticsearch · GitHub
Thanks in advance.