Banging my head on this application logs with APM

Elastic Observability
1

Hi,
I am setting up a Docker based test environment with Elasticsearch, Kibana, Fleet and spring-boot with APM. I can see transactions with a lot of information about requests. I can monitor exceptions. But I just cannot find a way to configure the application logs to be send to Elasticsearch as well.

Would love to use this option: log_ecs_reformatting=SHADE

But I just cannot work with it. Spring boot app running local on the Mac, or in a Docker container as well. Any tips for debugging are more than welcome.

Thanks,

Jettro

2

Hi @jettro

What version of the stack?

When you run local on the mac can you find the logs written by the app?

If so if you are running on a mac, just install filebeat and setup filestream to point to the logs and point to elasticsearch and filebeat will ship the logs...

3

Thank you for thinking with me. I am running 8.5.0, and yes I could send the logs using filebeat. But the intention of my experiment is to understand running apm/fleet with a number of spring boot apps that we want to run in a Docker (later Kubernetes) environment. I understood I could use APM with Fleet to send the logs to Elasticsearch. So that is what I am trying to do. Running the spring boot application direct on the mac is just to try out configs.

4

Can you show me where you got that understanding from... Elastic Java APM agent does not ship application logs ... it ships Transactions, Traces and Metrics for Latency, Throughput, JVM Metrics etc...

Perhaps I am missing something, application logs are shipped by a different integration with agent that would typically be the custom logs integration.

Unfortunately I think the term "Agent" is a bit overloaded....

There is the Java APM Agent that ships APM Data

Then there is the elastic Agent that works with fleet which has 100s of integrations, one of which is custom logs... like spring boot logs.

If you have some other way to directly ship application logs to Elasticsearch you can use that as long as you are setting the correlation data.

5

You can send logs to APM, but I don't think it's supported by Elastic APM products. To do it you should be able to use OpenTelemetry logging (OpenTelemetry integration | APM User Guide [8.5] | Elastic). Send logs through APM Agents · Issue #693 · elastic/apm · GitHub appears to be the issue for tracking log support from Elastic APM products.

6

Thank you again, @stephenb. After reading the docs you pointed to again, I finally got it. I was reading it with an idea of how I thought it worked. Now I understand what the log integration does. It makes sense.

1 Like