GoLang APM agent panic runtime error: invalid memory address or nil pointer dereference

Elastic Observability APM
1

If you are asking about a problem you are experiencing, please use the following template, as it will help us help you. If you have a different problem, please delete all of this text :slight_smile:

Elasticsearch version: 7.6.0

APM Server version: 7.6.0

APM Agent language and version: GoLang Agent apm@v1.6.0

Original install method (e.g. download page, yum, deb, from source, etc.) and version:
We have been running GoLang Agent for a while but started throwing panic and makes my pod restart
.

Mar 4, 2020 @ 11:56:27.058 panic: runtime error: invalid memory address or nil pointer dereference - supplier-api
Mar 4, 2020 @ 11:56:27.058 [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8049fec] - supplier-api
Mar 4, 2020 @ 11:56:27.058 - supplier-api
Mar 4, 2020 @ 11:56:27.058 goroutine 6 [running]: - supplier-api
Mar 4, 2020 @ 11:56:27.058 runtime/internal/atomic.Xadd64(0xa0f405c, 0x149e02e, 0x0, 0xffffffff, 0x0) - supplier-api
Mar 4, 2020 @ 11:56:27.058 /usr/local/go/src/runtime/internal/atomic/asm_386.s:105 +0xc - supplier-api
Mar 4, 2020 @ 11:56:27.058 go.elastic.co/apm.(*breakdownTiming).accumulate(0xa0f405c, 0x149e02e, 0x0, 0x1, 0x0, 0x0, 0x0, 0x0) - supplier-api
Mar 4, 2020 @ 11:56:27.058 /go/pkg/mod/go.elastic.co/apm@v1.6.0/breakdown.go:167 +0x4e - supplier-api
Mar 4, 2020 @ 11:56:27.058 go.elastic.co/apm.(*breakdownMetricsMap).record(0xa0664e0, 0x841f1f7, 0x7, 0xa4a4220, 0x19, 0x841d4b9, 0x3, 0x0, 0x0, 0x149e02e, ...) - supplier-api
Mar 4, 2020 @ 11:56:27.058 /go/pkg/mod/go.elastic.co/apm@v1.6.0/breakdown.go:230 +0x1b1 - supplier-api
Mar 4, 2020 @ 11:56:27.058 go.elastic.co/apm.(*breakdownMetrics).recordTransaction(0xa066540, 0xa06c1c0, 0x0) - supplier-api
Mar 4, 2020 @ 11:56:27.058 /go/pkg/mod/go.elastic.co/apm@v1.6.0/breakdown.go:198 +0x17d - supplier-api
Mar 4, 2020 @ 11:56:27.058 go.elastic.co/apm.(*Tracer).loop(0xa00a0f0) - supplier-api
Mar 4, 2020 @ 11:56:27.058 /go/pkg/mod/go.elastic.co/apm@v1.6.0/tracer.go:841 +0x1c56 - supplier-api
Mar 4, 2020 @ 11:56:27.058 created by go.elastic.co/apm.newTracer - supplier-api
Mar 4, 2020 @ 11:56:27.058 /go/pkg/mod/go.elastic.co/apm@v1.6.0/tracer.go:410 +0x615

Provide logs and/or server output (if relevant):
Importing and Instrumenting the GoLang APM Agent

import (
"context"

"github.com/lib/pq"
"go.elastic.co/apm"
"go.elastic.co/apm/module/apmsql"
_ "go.elastic.co/apm/module/apmsql/pq"

)

/main function
func main() {
fmt.Println("Main started")
//Set up router and handler
router := mux.NewRouter()
apmgorilla.Instrument(router)
controller := controllers.Controller{}
router.HandleFunc....
fmt.Println("Listening on port 8000")
http.ListenAndServe(":8000", apmhttp.Wrap(router))
}

Then create custom spans in each function

//Custom Span for Elastic APM
span, ctx := apm.StartSpan(ctx, "ValidateAvailabilityRequest", "custom")
if span != nil {
	defer span.End()
}
2

@marv72 sorry you're experiencing this, and thanks for all the details. It appears to be an issue specifically with 386 systems.

I've reproduced the bug, and created an issue to track it here: https://github.com/elastic/apm-agent-go/issues/727. Please subscribe to the issue for updates.

For the short term, you can set the environment variable ELASTIC_APM_BREAKDOWN_METRICS=false, which will disable the "Time spent by span type" graph data.

3

This is now fixed in the just-released v1.7.1. If you continue to have problems, please reopen the GitHub issue.

4

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.