We are using the APM Java Agent (apm-agent-attach:1.36.0) for a Spring Boot application. We have set the log_sending to true. The logs are sent to the APM Server and are visible in the Kibana. Everything works fine.
But we have two log appenders defined inside our Spring Boot Application. A console appender and a rolling file appender. The APM Java Agent sends the logs from both appenders to Elasticsearch. So we have all log messages twice. Once from the console appender, once from the RollingFileAppender.
Is it somehow possible to tell the APM Agent which appender to take?
Hi @SimonS1 , welcome to the forum and thanks for your feedback on this feature.
There is currently no option to select which appenders are used to send logs, the agent will send them all, so in your case it means there is some duplication as the log messages are written to both console and a log file.
However, each appender is using a distinct value for event.dataset that is using <app-name>.<appender-name> where <app-name> is the application name in agent configuration and <appender-name> the name of the appender in your logger configuration.
So, for now the simplest option is to define an ingest pipeline that will discard the log messages before they are stored.
See documentation on how to configure this. I am not very familiar with ingest pipelines, but should be quite straightforward if you already use them in Elasticsearch.
Currently only the Java agent has this log_sending feature available, and we are currently gathering feedback (like yours) on this feature, this will likely be something we improve in the future.