Kibana version:
v8.3.1
Elasticsearch version:
v8.3.1
APM Server version:
APM Server version 8.3.1
APM Agent language and version:
elastic-apm==6.10.1 using Python 3.8
Browser version:
Google Chrome Versão 103.0.5060.114
Original install method (e.g. download page, yum, deb, from source, etc.) and version:
Docker Layer
Is there anything special in your setup?
Using the default values.
Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
I'm using AWS Lambda Container Python image based and using SAM to deploy my Function that has S3 trigger.
I'm using the decorator "@capture_serverless()" on my main handler() function, but I'm not able to see ANY transaction, error or anything on my service.
The weird thing is that I'm able to see my service "lambda-vda" on Kibana, but always empty.
Steps to reproduce:
Dockerfile with the following lines at top:
FROM docker.elastic.co/observability/apm-lambda-extension-x86_64:latest AS lambda-extension
FROM public.ecr.aws/lambda/python:3.8
COPY --from=lambda-extension /opt/elastic-apm-extension /opt/extensions/elastic-apm-extension
all my related function stuff being instaled here, like TESSERACT and another python packages.
Provide logs and/or server output (if relevant):
My enviroment variables in the template;yaml SAM file:
Environment:
Variables:
ELASTIC_APM_LAMBDA_APM_SERVER: https://*.apm.eastus2.azure.elastic-cloud.com:443
ELASTIC_APM_SECRET_TOKEN: ****
ELASTIC_APM_LOG_LEVEL: trace
ELASTIC_APM_DATA_RECEIVER_TIMEOUT_SECONDS: 60
ELASTIC_APM_DATA_FORWARDER_TIMEOUT_SECONDS: 15
ELASTIC_APM_SERVICE_NAME: lambda-vda
So here I will share some logs that I'm able to see on my Cloudwatch logs related to some Lambda event:
{
"log.level": "debug",
"@timestamp": "2022-07-11T14:56:25.759Z",
"log.origin": {
"file.name": "apm-lambda-extension/main.go",
"file.line": 70
},
"message": "Register response: {\n\t\"functionName\": \"vda\",\n\t\"functionVersion\": \"$LATEST\",\n\t\"handler\": \"\"\n}",
"ecs.version": "1.6.0"
}
I see A LOT of warnings like this
[DEBUG] 2022-07-11T16:51:51.584Z 5e8b707b-7833-4b14-829c-5fa12ddde65f Skipping instrumentation of redis. Module redis.client not found
{
"log.level": "info",
"@timestamp": "2022-07-11T14:56:25.784Z",
"log.origin": {
"file.name": "logsapi/subscribe.go",
"file.line": 122
},
"message": "Extension listening for Lambda Logs API events on 169.254.79.130:43049",
"ecs.version": "1.6.0"
}
{
"log.level": "info",
"@timestamp": "2022-07-11T14:56:25.789Z",
"log.origin": {
"file.name": "apm-lambda-extension/main.go",
"file.line": 112
},
"message": "Waiting for next event...",
"ecs.version": "1.6.0"
}
After a log showing "received event.":
{
"log.level": "debug",
"@timestamp": "2022-07-11T14:56:32.562Z",
"log.origin": {
"file.name": "apm-lambda-extension/main.go",
"file.line": 126
},
"message": "{\n\t\"eventType\": \"INVOKE\",\n\t\"deadlineMs\": 1657551692561,\n\t\"requestId\": \"2b89ce98-84bd-4eea-91bb-9fae8faa9edf\",\n\t\"invokedFunctionArn\": \"arn:aws:lambda:us-east-1:
**:function:vda\",\n\t\"tracing\": {\n\t\t\"type\": \"X-Amzn-Trace-Id\",\n\t\t\"value\": \"Root=1-62cc3a19-0ecf8a350f40e8b406617329;Parent=5797d681116f11e8;Sampled=0\"\n\t}\n}",
"ecs.version": "1.6.0"
}
{
"log.level": "debug",
"@timestamp": "2022-07-11T14:56:42.762Z",
"log.origin": {
"file.name": "extension/apm_server_transport.go",
"file.line": 182
},
"message": "Transport status set to healthy",
"ecs.version": "1.6.0"
}
{
"log.level": "debug",
"@timestamp": "2022-07-11T14:56:52.511Z",
"log.origin": {
"file.name": "extension/apm_server_transport.go",
"file.line": 200
},
"message": "APM server Transport status set to Healthy",
"ecs.version": "1.6.0"
}
{
"log.level": "debug",
"@timestamp": "2022-07-11T14:56:52.511Z",
"log.origin": {
"file.name": "extension/apm_server_transport.go",
"file.line": 183
},
"message": "APM server response body: ",
"ecs.version": "1.6.0"
}
{
"log.level": "debug",
"@timestamp": "2022-07-11T14:57:33.166Z",
"log.origin": {
"file.name": "apm-lambda-extension/main.go",
"file.line": 176
},
"message": "Received agent done signal",
"ecs.version": "1.6.0"
}
{
"log.level": "debug",
"@timestamp": "2022-07-11T14:57:33.166Z",
"log.origin": {
"file.name": "logsapi/subscribe.go",
"file.line": 178
},
"message": "Current invocation over. Interrupting logs processing goroutine",
"ecs.version": "1.6.0"
}
{
"log.level": "debug",
"@timestamp": "2022-07-11T14:57:33.177Z",
"log.origin": {
"file.name": "extension/apm_server_transport.go",
"file.line": 101
},
"message": "Flush started - Checking for agent data",
"ecs.version": "1.6.0"
}
{
"log.level": "debug",
"@timestamp": "2022-07-11T14:57:33.177Z",
"log.origin": {
"file.name": "extension/apm_server_transport.go",
"file.line": 110
},
"message": "Flush ended - No agent data on buffer",
"ecs.version": "1.6.0"
}
{
"log.level": "debug",
"@timestamp": "2022-07-11T15:03:20.757Z",
"log.origin": {
"file.name": "apm-lambda-extension/main.go",
"file.line": 126
},
"message": "{\n\t\"eventType\": \"SHUTDOWN\",\n\t\"deadlineMs\": 1657551802714,\n\t\"requestId\": \"\",\n\t\"invokedFunctionArn\": \"\",\n\t\"tracing\": {\n\t\t\"type\": \"\",\n\t\t\"value\": \"\"\n\t}\n}",
"ecs.version": "1.6.0"
}
I'm not able to find in the logs any error or a message the seems an error.
I use the same credentials in another elastic-apm integration with Django without a problem.
