Unable to see the spring boot application logs in ElasticCloud APM

If you are asking about a problem you are experiencing, please use the following template, as it will help us help you. If you have a different problem, please delete all of this text :slight_smile:

TIP 1: select at least one tag that further categorizes your topic. For example server for APM Server related questions, java for questions regarding the Elastic APM Java agent, or ui for questions about the APM App within Kibana.

TIP 2: Check out the troubleshooting guide first. Not only will it help you to resolve common problems faster but it also explains in more detail which information we need before we can properly help you.

Kibana version:
7.17.6

Elasticsearch version:
7.17.6

APM Server version:
7.17.6

APM Agent language and version:
Java 11

Browser version:
Chrome 113

Original install method (e.g. download page, yum, deb, from source, etc.) and version:
Elastic Cloud

Fresh install or upgraded from other version?
Elastic CLoud

Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc.

No Load balancers setup

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):

Written Spring Boot Application service using Java. Building the application through Maven.

  1. I used APM library and logback encoder in depencies in Maven
    </>

    co.elastic.apm
    apm-agent-attach
    1.20.0


    co.elastic.logging
    logback-ecs-encoder
    1.5.0

    </>
  1. We have build the properties to read the APM Configurations like this
    </>
    elastic.apm.enabled=true
    elastic.apm.server_url =
    elastic.apm.service_name =customer-insights
    elastic.apm.secret_token =
    elastic.apm.environment =dev
    elastic.apm.application_packages =com.search.customerinsights.*
    elastic.apm.log_level =DEBUG
    elastic.apm.log_sending =true
    elastic.apm.enable_log_correlation =true
    elastic.apm.log_ecs_reformatting =OVERRIDE
    </>

  2. We can see the service in the APM in our dev cluster.

  3. But when we see the logs tab, it says there are no log messages to display.
    image

  4. In java application code we are using slf4j logger like this
    </>
    import lombok.extern.slf4j.Slf4j;

@Slf4j
public class CustomerInsightsController {
log.info("CustomerInsightsController - getCustomerInsights: for a given user : " + request.getUser_id());
}
</>

Steps to reproduce:

  1. Build the sample application
  2. Load the APM
  3. Connect to the APM server and integrate the logs.

Do we need to setup the filebeat? I'm trying to setup the filebeat also.
I'm configuring file beat. We are using Elastic Cloud and set the cloud.id and cloud.auth keys in filebeat.yml file but giving authentication error.

Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at https://sakjdfhaslkdfjhasdflkjashdflkasjdhf0.us-east-1.aws.found.io:443: 401 Unauthorized: {"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate user [elasticclient] for REST request [/]","header":{"WWW-Authenticate":["Basic realm="security" charset="UTF-8"","Bearer realm="security"","ApiKey"]}}],"type":"security_exception","reason":"unable to authenticate user [elasticclient] for REST request [/]","header":{"WWW-Authenticate":["Basic realm="security" charset="UTF-8"","Bearer realm="security"","ApiKey"]}},"status":401}]

Whether you want to use logging to files + filebeat to ship your logs or the APM agent based log sending depends on your requirements. See this page for a comparison. To get running quickly, I would recommend you to start with agent based log sending.

In order to see why your APM agent is currently not sending the looks we will need the debug logs from the agent. Please follow this procedure and provide us the debug logs.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.