Elastic APM 8.2, sourcemap not working, "unable to find sourcemap.url"

[mkf1 (Kasper)]

Elastic APM 8.2, sourcemap does not work.

Sourcemaps uploaded and saved in Kibana without issues.
Service version, service name looks good.

But it still always complains about "sourcemap.url not find":

Any idea why I cant get this to work?

Kibana version:
8.2.1

Elasticsearch version:
8.2.1

APM Server version:
8.2.1 (Elastic APM (Elastic Agent))

APM Agent language and version:

Browser version:

Original install method (e.g. download page, yum, deb, from source, etc.) and version:
Docker. Elasticsearch (8.2.1), Kibana (8.2.1), Fleet-Server (8.2.1) and Elastic APM (8.2.1) running as containers.
EDIT
All services running as containers on the same host.

EDIT2:

Here is the out from "elastic-agent inspect" on the APM instance/server:

Note: the URL for the fleet host, https://elastic-monitoring-fleet.<redacted>:9200 is going to reverse proxy (nginx) which is in-front the fleet server.

If anyone knows where I can see more information about the error I would like to know. I have checked multiple logs on the different services but can't find anything related to the souremap error.

[Alberto_Delgado]

Hi @mkf1,

Thanks for reaching out!

Could you have a look at this?

Also recommended to check this for guidance

Thanks,
Alberto

[mkf1 (Kasper)]

Hi @Alberto_Delgado

Thank you.
Yes I have seen that github issue. But we are not using relavete path in bundle_filepath, we have absolute path.

In the first screenshot in my post you can see that the bundle_filepath has the absolue path, e.g. https://<redacted>/995.4364f50e3faea049cebb.js

We have a simple powershell script which is using curl to upload the sourcemaps.

$files = Get-ChildItem -PATH $path -Recurse -Filter '*.map'

foreach ($file in $files)
{
    $fullname = $file.FullName
    $relativeSourceMapPath = $fullname.Replace($path, '')
    $relativeSourcefilePath = $relativeSourceMapPath.replace('.map', '')
    $relativeSourcefilePathURL = $relativeSourcefilePath.replace('\', '/')
    $absoluteSourcePath = "https://$iisHost" + $relativeSourcefilePathURL
    
    $CurlArguments = "-X", "POST",
        "$kibanaServerUrl/api/apm/sourcemaps",
        "-H", "Content-Type: multipart/form-data",
		"-H", "kbn-xsrf: true",
		"-H", "Authorization: Basic $kibanaBasicAuth",
        "--form", "service_name=$apmServiceName",
        "--form", "service_version=$version",
        "--form", "bundle_filepath=$absoluteSourcePath",
        "--form", "sourcemap=@$fullname",
        "--output", "nul",
        "--insecure",
        "--silent",
        "--show-error",
        "--fail"
        
	Write-Host "---"
    Write-Host "File: $absoluteSourcePath"
	Write-Host "File: $fullname"
    
    & $CurlExecutable @CurlArguments > $null 2>&1
}

This is working fine, we can't see any issue with how the sourcemaps are being uploaded and when we look at the Kibana /api/apm/sourcemaps endpoint the sourcemaps looks correct.

If there is any other log or anything I can look at to see why it fails with the "unable find sourcemap.url" error please let me now. I have checked all relveant logs that I know of in my elastic stack and can't see anything related to the sourcemap error.

[mkf1 (Kasper)]

I have done two edits to my post and added some more information about setup and the output of "elastic-agent inspect" from the apm instance/container.

[mkf1 (Kasper)]

I can see that it is this code that is throwing the "unable find sourcemap.url" here apm-server/fleet.go at 077d179ddff4d1bc5585a1b92d36ba068b02e585 · elastic/apm-server · GitHub .

If I understand correctly the apm-server is fetching the sourcemaps from the fleet-server via http request.
Then I would expect to see entries in my http access log for my fleet-server ( https://elastic-monitoring-fleet.<redacted>:9200), but there is no entries at all in the access log about http requests on /api/fleet/artifacts from apm-server.

[mkf1 (Kasper)]

Hi @Alberto_Delgado
I have figured out my issue
I have created a github issue about it here APM stacktrace doesn't work when more than 20 sourcemaps have been uploaded to /api/apm/sourcemaps · Issue #9372 · elastic/apm-server · GitHub