AWS Lambda Logs

Collect logs from your AWS Lambda functions and send them to Middleware without changing your function code. The OpenTelemetry Collector Lambda layer runs as a Lambda extension. It subscribes to the Lambda Telemetry API to receive everything your function writes to stdout and stderr, plus platform logs, and exports them to Middleware over OTLP.

1Lambda function ──(stdout/stderr)──► Telemetry API ──► OTel Collector extension ──(OTLP)──► Middleware

To also collect traces from your Lambda functions, follow the AWS Lambda APM guide. You can use the same Collector layer and collector.yaml for both.

Prerequisites#

  • An AWS account with permissions to manage Lambda functions, layers, and environment variables.
  • Your Middleware UID (https://<uid>.middleware.io) and API key (MW_API_KEY).

Setup#

1 Add the OpenTelemetry Collector layer#

Go to the opentelemetry-lambda releases page and copy the latest Collector layer ARN. The layers are published per region by AWS account 184161586896:

1arn:aws:lambda:<aws-region>:184161586896:layer:opentelemetry-collector-<amd64|arm64>-<version>:<layer-version>
  • Replace <aws-region> with your function's region.
  • Replace <amd64|arm64> with your function's architecture.
  • Replace <version> and <layer-version> with the values from the release.

In the AWS Console, go to Lambda → Functions → your function → Layers → Add a layer, choose Specify an ARN, paste the ARN, and click Add.

2 Create the collector configuration#

Create a file named collector.yaml in the root of your function's deployment package:

1receivers:
2  telemetryapi:
3
4processors:
5  batch:
6  decouple:
7  resource:
8    attributes:
9      - key: mw.account_key
10        value: <MW_API_KEY>
11        action: upsert
12      - key: service.name
13        value: "my-lambda-function"
14        action: upsert
15      - key: deployment.environment
16        value: "production"
17        action: upsert
18
19exporters:
20  otlp:
21    endpoint: https://<MW_UID>.middleware.io:443
22    headers:
23      authorization: <MW_API_KEY>
24
25service:
26  pipelines:
27    logs:
28      receivers: [telemetryapi]
29      processors: [resource, batch, decouple]
30      exporters: [otlp]

Replace:

  • <uid> with your Middleware project UID.
  • <MW_API_KEY> with your Middleware API key.
  • my-lambda-function and production with your service name and environment.

Keep the service.name attribute. Middleware rejects logs that do not identify a resource (such as service.name or host.id).

The decouple processor must be the last processor in the pipeline. It lets the function return without waiting for the export to finish.

Store your API key in AWS Secrets Manager or an encrypted environment variable instead of in plain text. The collector can read environment variables with ${env:MW_API_KEY}.

3 Point the layer to the configuration#

Go to Configuration → Environment variables and add:

1OPENTELEMETRY_COLLECTOR_CONFIG_URI=/var/task/collector.yaml

/var/task is the root of your deployment package.

4 Deploy and invoke the function#

Deploy the function with collector.yaml included, then invoke it. Logs appear in the Logs section of your Middleware account.

Limitations#

  • Logs are delayed. The decouple processor sends buffered logs at the start of the next invocation, or when the execution environment shuts down. A function that is invoked rarely may need a second invocation before its logs appear.
  • Remote configuration adds cold-start latency. OPENTELEMETRY_COLLECTOR_CONFIG_URI also supports s3:// and https:// URIs, but loading the file from your deployment package is faster.

Troubleshooting#

  • No logs in Middleware: check that OPENTELEMETRY_COLLECTOR_CONFIG_URI is set and that collector.yaml is at the root of the deployment package.
  • Extension fails to start: check the function's CloudWatch Logs for errors from the collector extension.
  • Layer errors: make sure the layer ARN matches your function's region and architecture (amd64 or arm64).
  • Deprecation warning for otlp: Collector layers built on collector v0.145.0 or later name the gRPC exporter otlp_grpc. otlp still works as a deprecated alias. To remove the warning, rename the exporter to otlp_grpc in exporters and in the pipeline.

Need assistance or want to learn more about Middleware? Get in touch with us via our Contact Us or join our Slack channel.