Log correlation with Log4j2 and AsyncLoggerContextSelector

kelunik · April 24, 2023, 12:07pm

If I use async logging, the transaction.id and trace.id fields aren't set in the log messages.

System.setProperty("log4j2.contextSelector", "org.apache.logging.log4j.core.async.AsyncLoggerContextSelector");

I'd like to keep using async logging, but still benefit from the trace and transaction correlation. What's the recommended way to do that?

kelunik · April 26, 2023, 8:15am

These seem to be the relevant parts:

github.com

elastic/apm-agent-java/blob/058b2679b8fdfa2482fc2738090ca597fa6710a1/apm-agent-plugins/apm-logging-plugin/apm-log4j2-plugin/src/main/java/co/elastic/apm/agent/log4j2/correlation/Log4j2LogCorrelationInstrumentation.java#L62


      
          @Override
          public ElementMatcher.Junction<ClassLoader> getClassLoaderMatcher() {
              return not(isBootstrapClassLoader())
                  // Do not instrument the agent log4j2 lib
                  .and(not(CustomElementMatchers.isAgentClassLoader()))
                  .and(classLoaderCanLoadClass("org.apache.logging.log4j.core.impl.LogEventFactory"));
          }
          
          @Override
          public ElementMatcher<? super NamedElement> getTypeMatcherPreFilter() {
              return nameEndsWith("LogEventFactory");
          }
          
          @Override
          public ElementMatcher<? super TypeDescription> getTypeMatcher() {
              return hasSuperType(
                  named("org.apache.logging.log4j.core.impl.LogEventFactory")
                      .or(named("org.apache.logging.log4j.core.impl.LocationAwareLogEventFactory"))
              );
          }

https://logging.apache.org/log4j/2.x/manual/extending.html#LogEventFactory

Note: When log4j is configured to have all loggers asynchronous, log events are pre-allocated in a ring buffer and the LogEventFactory is not used.

Elastic instruments the LogEventFactory, but this factory isn't used if async logging is used.

Jack_Shirazi · April 26, 2023, 1:56pm

Yes that's the instrumentation applied for Log4j. I think you're right, I can't see where we would capture async logs. I'll discuss with the team next week but I don't think I have a useful answer for you now

kelunik · April 26, 2023, 2:15pm

@Jack_Shirazi I've now added this to our already existing ContextDataProvider, see Elastic APM · GitHub

It seems like this might be a good API to use for the agent as well: Log4j – Extending Log4j 2

I haven't anything for error.id, yet.

Jack_Shirazi · May 3, 2023, 12:22pm

Thanks for the report and the suggestions, we've opened an issue and hope to get to it next Q

kelunik · May 3, 2023, 12:31pm

Thanks!

system · May 24, 2023, 8:32am

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.