Golang apmchi library question

[Evan_Something]

Kibana version: 7.4.2

Elasticsearch version: 7.4.2

APM Server version: 7.4.2

APM Agent language and version: unsure

Original install method (e.g. download page, yum, deb, from source, etc.) and version: docker

Fresh install or upgraded from other version? fresh install

Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc.

attempting a barebones setup...

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):

So I am trying to setup my first proof of concepts with APM. I have an API called Gateway that receives a request, and calls another Go service called Template Service.

I imported in my routes.go the "go.elastic.co/apm/module/apmchi" package, and I added the following line for instance in declaring routes for the template service:

func NewRoutes() *router.Mux {
	r := router.NewRouter()
	r.Use(middleware.Logger)
	r.Use(apmchi.Middleware())

	r.Route("/v1", func(r router.Router) {
		r.Get("/product-templates", getProductTemplates)
	})
	return r
}

I have similar in the Gateway project.

When I run both of them, they both appear in the APM console as services, and when I send a request that gets routed through the gateway to the template service, they appear as two separate and complete transactions.

Am I supposed to do additional setup? I wasn't sure if apmchi managed incoming requests, and there was brief instructions for setting it up.

To confirm, when i open APM, I see both services listed, and if I click Gateway, I see under Transactions a line item for:

GET /v1/product-templates

And if I go back and select Template service, i see an identical:

GET /v1/product-templates

Finally, to add a little more context, in the gateway service, I declare routes like such:

// NewRoutes returns the applications routes
func NewRoutes() *router.Mux {
	r := router.NewRouter()

	r.Use(middleware.Logger)
	r.Use(newCors().Handler)
	r.Use(apmchi.Middleware())

	r.Route("/v1", func(r router.Router) {
		r.Group(func(r router.Router) {
			user.Routes(r)
		})
		r.Group(func(r router.Router) {
			salestax.Routes(r)
		})
		r.Group(func(r router.Router) {
			template.Routes(r)

And i have a templates.go file to handle this route, with a setup like such:

func Routes(r router.Router) {
	r.Use(middleware.Authenticator)

	r.Get("/product-templates", getProductTemplates)
	r.Get("/themes", getThemes)
}

Which does a call to the Template service like such:

func getProductTemplates(w http.ResponseWriter, r *http.Request) {
	fmt.Println("gateway")
	resp, err := request.Get(url(fmt.Sprintf("/product-templates")))
	if err != nil {
		errors.NewHTTP(http.StatusInternalServerError, err.Error()).Response(w)
		return
	}

	w.WriteHeader(resp.Response().StatusCode)
	w.Write(resp.Bytes())
}

Any assistance in understanding is appreciated

[axw (Andrew Wilkins)]

Hi @Evan_Something, thanks for checking out APM!

You've already gotten most of the way, the final missing piece is to propagate the trace context. There's a couple of things you'll need to do to achieve this:

1. instrument the HTTP client in the gateway service
2. pass the incoming request context to the outgoing request

The first part can be done using apmhttp.WrapClient, e.g.:

var httpClient = apmhttp.WrapClient(http.DefaultClient)

You should then use this instrumented httpClient to perform the outgoing requests. It'll only create spans if the outgoing request context contains a transaction. You can achieve that like this:

func getProductTemplates(w http.ResponseWriter, r *http.Request) {
    fmt.Println("gateway")
    req, _ := http.NewRequestWithContext(r.Context(), "GET", url("/product-templates"), nil)
    resp, err := httpClient.Do(req)
    if err != nil {
        errors.NewHTTP(http.StatusInternalServerError, err.Error()).Response(w)
        return
    }
    w.WriteHeader(resp.Response().StatusCode)
    w.Write(resp.Bytes())
}

You might also like to use https://golang.org/x/net/context/ctxhttp to cut down on boilerplate a little bit. See the module/apmhttp docs for an example.