Missing Service Map when integrating Elastic APM with AWS Lambda?

[Tosho_Toshev]

Hello,

I'm new to Elastic APM and I'm trying to integrate it with AWS Lambda. I followed the steps described here: Monitoring AWS Lambda

I added the necessary layers and environment variables to my lambda functions. After that I created a basic step function and executed it several times. The execution was successful and I got the APM data to my trial elastic cloud instance.

The problem that I encountered is that the Service Map is missing for the lambda functions. I noticed that when the lambda function is calling an external server via http request the map is present, but the communication between the lambdas in the step function is not.

I went through the documentation and I saw that the Service Map is build with the help of the trace parent header. My guess is that I'm not seeing the Service Map between the lambda functions, because the lambda api invokes them via events and not http calls.

Is my guess correct or is there another reason that I'm not seeing the service map? Maybe I missed something in the documentation?

[Jack_Shirazi]

That sounds right, I haven't looked at lambda's invoking other lambdas. Can you detail exactly how you're invoking, so I can check if it's a bugfix or an enhancement needed?

[Tosho_Toshev]

Hello,

Sorry for the late reply!

I'm using a basic step function with a state machine that I created in the UI to orchestrate the lambda functions. Here's the code for it:

{
  "Comment": "Lambda APM test state machine",
  "StartAt": "File Upload",
  "States": {
    "File Upload": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "OutputPath": "$.Payload",
      "Parameters": {
        "Payload.$": "$",
        "FunctionName": "myFunctionName"
      },
      "Retry": [
        {
          "ErrorEquals": [
            "Lambda.ServiceException",
            "Lambda.AWSLambdaException",
            "Lambda.SdkClientException",
            "Lambda.TooManyRequestsException"
          ],
          "IntervalSeconds": 2,
          "MaxAttempts": 6,
          "BackoffRate": 2
        }
      ],
      "Next": "Anchor Upload"
    },
    "Anchor Upload": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "OutputPath": "$.Payload",
      "Parameters": {
        "Payload.$": "$",
        "FunctionName": "myFunctionName"
      },
      "Retry": [
        {
          "ErrorEquals": [
            "Lambda.ServiceException",
            "Lambda.AWSLambdaException",
            "Lambda.SdkClientException",
            "Lambda.TooManyRequestsException"
          ],
          "IntervalSeconds": 2,
          "MaxAttempts": 6,
          "BackoffRate": 2
        }
      ],
      "End": true
    }
  }
}

This is how it looks visually:

From here I just pass a simple json with a string representing the file name. I don't do anithing with the file, I just log the file name in the two lambda functions.

[Jonas_Kunz]

Unfortunately step functions are not supported. Therefore you are spot on with your analysis:

I went through the documentation and I saw that the Service Map is build with the help of the trace parent header. My guess is that I'm not seeing the Service Map between the lambda functions, because the lambda api invokes them via events and not http calls.

Technically, the step function invokes the individual steps and just passes the data between them.
Because step functions are not supported, the individual steps each have their own trace and therefore appear unrelated in elastic APM.