How can i configure log4j to use AsyncAppender instead SocketAppender?

Elastic Stack Elasticsearch
1

Hi guys,

I found this thread from 2015 Elasticsearch AsyncAppender configuration and this another from 19d ago Log4j2 AsyncAppender in Elasticsearch 5.x and my question is: It's possible to configure AsyncAppender on ES 5.x?

The configuration from the first link still valid?

2

Are you sure you want to do that?

Using AsyncAppender will cause logging messages to buffer into memory that goes straight to heap (iirc) and will eventually cause memory pressure issues (i.e. long GCs). Which is very problematic to overall cluster performance and stability.

Elasticsearch logging at INFO level is very quiet and should not be the cause of any I/O bottleneck.

1 Like
3

Thank you Thiago.

I've changed my logging configuration to INFO and i'll observe.

4

Hi! I posted the issue from 19 days ago. I believe we actually do need asynchronous appending:

We have a setup with Elasticsearch logging to a Redis queue. Suppose the ES loglevel would temporarily be changed to DEBUG (or maybe even TRACE) and there would also be network issues in between ES and Redis. If a synchronous appender would be used, then in this setup the logging would actually render the cluster to be very slow/unresponsive because it would have to wait for all logs to be processed, before getting to continue its normal operations.

Or is there something I'm missing/does ES solve this in a different way?

5

@tvlieg I understand what you mean and indeed enabling TRACE and DEBUG with synchronous may impose some performance impact but I don't think it pays off the performance issues you would have if async is used and heap gets flooded with logging messages because disk can not keep up with it.

Anyway, I have yet to see a situation that enabling DEBUG and TRACE in production with sync logging would severely impact cluster performance. When you enable that logging level it is generally because you have bigger issues going on.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.