Apm agent for jetty application not tracking transactions

Kibana version:
7.7.0

Elasticsearch version:
7.7.0

APM Server version:
7.7.0

APM Agent language and version:
java
1.16.0

Browser version:
Chrome

Original install method (e.g. download page, yum, deb, from source, etc.) and version:
Maven

Fresh install or upgraded from other version?
Fresh

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):

I setup basic Jetty server with AbstractHandler and started a simple server on port 9001. APM only tracking main class JVM and transactions/API are not tracked.

Steps to reproduce:

  1. Basic HTTP Jetty Server
  2. Connect APM to Process/Java
  3. API not tracked

Errors in browser console (if relevant):
No Error

** Jetty Server

package jetty;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.eclipse.jetty.server.Request;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.handler.AbstractHandler;

import co.elastic.apm.attach.ElasticApmAttacher;

public class Application extends AbstractHandler {
	public void handle(String target, Request baseRequest, HttpServletRequest request, HttpServletResponse response)
			throws IOException, ServletException {
		response.setContentType("text/html;charset=utf-8");
		response.setStatus(HttpServletResponse.SC_OK);
		baseRequest.setHandled(true);
		response.getWriter().println("<h1>Hello World</h1>");
	}

	public static void main(String[] args) throws Exception {
		
		ElasticApmAttacher.attach();
		Server server1 = new Server(9001);
		server1.setHandler(new Application());
		
		server1.start();
		server1.join();
	}
}

Provide logs and/or server output (if relevant):

2020-05-21 14:55:00.656:INFO::main: Logging initialized @450ms to org.eclipse.jetty.util.log.StdErrLog
2020-05-21 14:55:00.779:INFO:oejs.Server:main: jetty-9.4.28.v20200408; built: 2020-04-08T17:49:39.557Z; git: ab228fde9e55e9164c738d7fa121f8ac5acd51c9; jvm 1.8.0_232-b09
2020-05-21 14:55:00.839:INFO:oejs.AbstractConnector:main: Started ServerConnector@379619aa{HTTP/1.1, (http/1.1)}{0.0.0.0:9001}
2020-05-21 14:55:00.847:INFO:oejs.Server:main: Started @654ms
2020-05-21 14:55:37.408 [Attach Listener] INFO co.elastic.apm.agent.util.JmxUtils - Found JVM-specific OperatingSystemMXBean interface: com.sun.management.OperatingSystemMXBean
2020-05-21 14:55:37.720 [elastic-apm-server-healthcheck] INFO co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server is available: {  "build_date": "2020-05-12T00:04:54Z",  "build_sha": "64e91c95329991c36b16ff94fd34ea75230c06c2",  "version": "7.7.0"}
2020-05-21 14:55:37.754 [Attach Listener] INFO co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.16.0 as Test-Jetty2 on Java 1.8.0_232 (AdoptOpenJDK) Mac OS X 10.15.4
2020-05-21 14:55:37.769 [Attach Listener] INFO co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to RUNNING state
2020-05-21 14:55:37.918 [elastic-apm-remote-config-poller] INFO co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Received new configuration from APM Server: {}

Agent cannot establish connection with APM server. You need to setup one and make sure it is accessible to the agent.

Also, just out of curiosity, why did you choose this installation option rather than using the javaagent option?

Wrong logs uploaded. I have now updated correct java logs. I have connected via java agent only but still it doesn't show all transactions. Only JVM.

Since you extend org.eclipse.jetty.server.handler.AbstractHandler, HTTP requests are directly passed to your implementation, without any Servlet being involved. There is no out of the box support for this Jetty-specific API (check out the supported technologies page, which says we only support ServletContextHandler).
What is the use case you are testing? Is your real app serving HTTP requests similarly without relying on Servlets?

In real app - it does spin up a server from that jetty application like I mentioned. This is the existing architecture of the application - API gateway!

Regards,

Abhishek Tanwar

Thanks for explaining.

An immediate way to get that traced is by using our public API - simply add a dependency in the agent's API jar and annotate your handle method with our @CaptureTransaction annotation. You can use the annotation attributes to set the transaction's name and type, and you can further decorate it in your code by getting it through currentTransaction and adding labels or custom context.

In the future, we may add out-of-the-box support for additional Handlers. In your case, are you only interested in tracing custom handlers (like the one demonstrated in the code above), or are there existing Jetty handlers you would be happy to see traced?