AWS Lambda

This guide walks you through instrumenting AWS Lambda with OpenTelemetry and Middleware. These instructions can also be found on the Installation page in your Middleware Account.

Introduction

OpenTelemetry provides a standardized way to instrument applications for collecting telemetry data. However, implementing OpenTelemetry in AWS Lambda presents unique challenges due to the serverless nature of the environment. This guide aims to create a seamless OpenTelemetry experience for Lambda users for different programming languages

Key Challenges

Early Function Exit: Lambda functions may exit before traces can be exported.
Container Freezing: AWS Lambda may freeze the container if no new requests arrive, suspending all processes.
Performance Concerns: The function should remain as lightweight as possible for optimal Lambda performance.

-- START OF NEW DOC --

Prerequisites

Node.js

Python

.NET

Node.js 18+: check version with node -v

Python 3.8+: check with python -V

.NET 6+: check version with dotnet --version

Features Supported

Traces	Metrics	App Logs	Custom Logs	Profiling
✅	✅	✅	✅	✖

.NET does not Support Application logs by default.

Installation

Node.js

Python

.NET

Copy the ARN for the OpenTelemetry (OTel) Collector Lambda Layer:
1. Navigate to the OpenTelemetry Lambda Repository and locate the appropriate layer for your environment.
2. Copy the Amazon Resource Name (ARN) and change the <region> tag to the region your Lambda is in.
```
arn:aws:lambda:<region>:184161586896:layer:opentelemetry-nodejs-0_9_0:4
```
Copy the ARN for the OTel Auto Instrumentation Layer:
1. Navigate to the OpenTelemetry Lambda Repository and locate the appropriate layer for your environment.
2. Copy the ARN and and change the <region> tag to the region your Lambda is in.
```
arn:aws:lambda:<region>:184161586896:layer:opentelemetry-nodejs-0_9_0:4
```
In the AWS Console navigate to Lambda → Functions → your specific Lambda function and create a Layer for each ARN above

The Layers above must be added in order - Collector then Auto Instrumentation - or they will not work.

Review the added layer and ensure it matches the ARN. Finally, click Add to complete the process.

Set the following environment variables in your Lambda function configuration:

AWS_LAMBDA_EXEC_WRAPPER=/opt/otel-handler
NODE_OPTIONS=--require ./lambda-config.js
OTEL_SERVICE_NAME=your-service-name
OTEL_EXPORTER_OTLP_ENDPOINT=https://<MW_UID>.middleware.io:443
OTEL_RESOURCE_ATTRIBUTES=mw.account_key=<MW_API_KEY>
OTEL_LAMBDA_DISABLE_AWS_CONTEXT_PROPAGATION=true
OTEL_PROPAGATORS=tracecontext

Create a file named lambda-config.js in your Lambda function's root directory with the following content:

const { SimpleSpanProcessor } = require("@opentelemetry/sdk-trace-base");
const {
  OTLPTraceExporter,
} = require("@opentelemetry/exporter-trace-otlp-proto");
global.configureTracerProvider = (tracerProvider) => {
  console.log("Configuring tracer provider with Simple Span Processor");
  const spanProcessor = new SimpleSpanProcessor(new OTLPTraceExporter());
  tracerProvider.addSpanProcessor(spanProcessor);
};

This configuration overrides the default Batch Span Processor with a Simple Span Processor, ensuring immediate processing and exporting of spans.

Copy the ARN for the OpenTelemetry (OTel) Collector Lambda Layer:
1. Navigate to the OpenTelemetry Lambda Repository and locate the appropriate layer for your environment.
2. Copy the Amazon Resource Name (ARN) and change the <region> tag to the region your Lambda is in.
```
arn:aws:lambda:<region>:184161586896:layer:opentelemetry-python-0_10_0:1
```
Copy the ARN for the OTel Auto Instrumentation Layer:
1. Navigate to the OpenTelemetry Lambda Repository and locate the appropriate layer for your environment.
2. Copy the ARN and and change the <region> tag to the region your Lambda is in.
```
arn:aws:lambda:<region>:184161586896:layer:opentelemetry-python-0_10_0:1
```
In the AWS Console navigate to Lambda → Functions → your specific Lambda function and create a Layer for each ARN above.

The Layers above must be added in order - Collector then Auto Instrumentation - or they will not work.

Review the added layer and ensure it matches the ARN. Finally, click Add to complete the process.

Set the following environment variables in your Lambda function configuration:

AWS_LAMBDA_EXEC_WRAPPER=/opt/otel-handler
OTEL_SERVICE_NAME=your-service-name
OTEL_EXPORTER_OTLP_ENDPOINT=https://<MW_UID>.middleware.io:443
OTEL_RESOURCE_ATTRIBUTES=mw.account_key=<MW_API_KEY>
OTEL_LAMBDA_DISABLE_AWS_CONTEXT_PROPAGATION=true
OTEL_PROPAGATORS=tracecontext

Import and set up a LoggerProvider to send log messages at desired log levels:

# Import packages
import logging
from opentelemetry._logs import set_logger_provider
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import SimpleLogRecordProcessor
from opentelemetry.exporter.otlp.proto.http._log_exporter import (
    OTLPLogExporter,
)

# Set the LoggerProvider

First you'll need to install the MW.APM package via CLI or Visual Studio.
1. For CLI run
```
dotnet add package MW.APM
```
2. For Visual Studio in Windows by searching for the MW.APM nuget package in Tools > Nuget Package Manager > Manage Nuget Packages for Solutions > Browse

Add the following to a your program.cs file:

var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
.AddEnvironmentVariables()
.Build();

builder.Services.ConfigureMWInstrumentation(configuration);

builder.Logging.AddConfiguration(configuration.GetSection("Logging"));
builder.Logging.AddConsole();

Add the below configs to your appsettings.json and then start your project. Note that ApiKey and TargetURL will be populated once logged in to the docs:

"MW": {
  "ApiKey": "<MW_API_KEY>",
  "TargetURL": "https://<MW_UID>.middleware.io:443",
  "ServiceName": "<service-name>",
  "ProjectName": "<project-name>",
  "ConsoleExporter": "true",
  "ExcludeLinks": "[\"https://localhost:3000/health\"]",
  "ApmCollectMetrics": "true",
  "ApmCollectTraces": "true",
  "ApmCollectLogs": "true"

If you want to know more about all 2 methods and its attributes. You can refer this link

This is a one-time configuration. After these configuration changes are made, each time the lambda function deployed, the .NET instrumentation will also run.

Verify Installation

Deploy and Test your application .Check your Middleware account for traces by navigating to APM -> Services and finding the name of your function in the services list.

X-Ray Tracing

Ensure X-Ray tracing is enabled for your Lambda function to get the full benefit of the instrumentation.

Sending Custom Data

To capture logic specific to your business you can use the OpenTelemetry API as follows:

Node.js

Python

const { trace } = require("@opentelemetry/api");

const tracer = trace.getTracer("{SERVICE-NAME}", "1.0.0");

exports.handler = async (event, context) => {
  const span = tracer.startSpan("lambdaHandler");

  try {
    span.setAttribute("event", JSON.stringify(event));

from opentelemetry import trace

# Acquire a tracer
tracer = trace.get_tracer("cars.tracer")

def fetch_cars():
	# Start a custom span
  with tracer.start_as_current_span("cars.fetch") as fetch_cars_span:
      # Set custom span attributes
      fetch_cars_span.set_attribute("cars.count", len(cars))

Troubleshooting

If you encounter missing spans:

Verify that all environment variables are correctly set.
Fore Node, check that the lambda-config.js file is in the root directory of your Lambda function.
Ensure that the OpenTelemetry layer is correctly added to your function.
Review CloudWatch logs for any error messages related to OpenTelemetry.

Need assistance or want to learn more about Middleware? Contact our support team in Slack.