Any plans to support java logback transport to es in java apm agent


(cai) #1

the java apm agent can support logback ,so we hope to see log in kibana

Now we use our own agent to put into es

azuze Application Insights now can support this

thanks


(Felix Barnsteiner) #2

Hi and thanks for your question!

The agent does not support sending the logs directly to Elasticsearch and we don't have immediate plans to do so.

What is supported is that the agent attaches the trace ids to the logs, so that you can correlate logs with traces.

Best practice is to log to a file and use Filebeat to send the logs to Logstash or an Elasticsearch ingest pipeline. You can either log as JSON (see this example setup), which makes parsing the logs easier or log in plain text and process the logs via a grok or dissect filter in Logstash or in an ingest pipeline.

One of the challenges in a design where the agent directly sends logs to Elasticsearch is how to accommodate for temporary network outages and how to handle the case when Elasticsearch is responding slowly because it is under heavy load. When logging to a file, you can just continue shipping the logs when the network is available again. But when directly sending logs from your application to Elasticsearch, you either have to buffer all logs in memory, which can lead to OutOfMemoryErrors, or you have to drop logs. Neither of those seems like a good idea.


(cai) #3

log is a part of apm
we hope only one agent collect our logging,metrics,trace
Is it possible elastic apm agent provide the logging plugin
In most cases, the log is allowed to be lost.