Otel Agent

An OpenTelemetry Collector receives application telemetry and exports it to Middleware using the OpenTelemetry Protocol (OTLP). This integration supports traces, metrics, and logs through the configured signal pipelines.

Use this integration if your applications are already instrumented with OpenTelemetry or you want to collect and process telemetry from multiple services through a centralized Collector.

How it works#

The OpenTelemetry Collector receives telemetry from applications, processes it, and exports it to Middleware.

The collection pipeline consists of three components:

• Receivers: Accept telemetry from OpenTelemetry SDKs and other supported sources using OTLP/gRPC or OTLP/HTTP.
• Processors: Batch, filter, enrich, or sample telemetry before exporting it.
• Exporters: Send processed traces, metrics, and logs to Middleware over HTTPS.

You can send telemetry directly from an application or route it through an OpenTelemetry Collector for centralized configuration and processing.

Prerequisites#

Before configuring OpenTelemetry, make sure you have:

• A Middleware account with an API key (MW_API_KEY) and target URL.
• An OpenTelemetry Collector installed, or an existing Collector deployment.
• Applications instrumented using OpenTelemetry SDKs or other supported telemetry sources.
• Outbound HTTPS access to your Middleware target on port 443.

Supported protocols#

Middleware supports traces, metrics, and logs through OTLP/gRPC and OTLP/HTTP.

Note: Use the root endpoint for OTLP/gRPC. For OTLP/HTTP, use the appropriate signal endpoint.

Installation#

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 512
    spike_limit_mib: 128
  batch: {}

exporters:
  otlphttp:
    endpoint: https://<MW_UID>.middleware.io:443
    headers:
      authorization: <MW_API_KEY>

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp]

    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp]

    logs:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp]

otelcol --config=config.yaml

otelcol-contrib --config=config.yaml

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc

export OTEL_SERVICE_NAME=my-service
export OTEL_RESOURCE_ATTRIBUTES="service.version=1.0.0,deployment.environment=production"
For OTLP/HTTP:
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

export OTEL_SERVICE_NAME=my-service

Alternative: Send telemetry directly to Middleware#

You can export telemetry directly from an OpenTelemetry SDK without deploying a separate Collector.

Configure the following environment variables in your application:

export OTEL_EXPORTER_OTLP_ENDPOINT=https://<MW_UID>.middleware.io:443
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
export OTEL_EXPORTER_OTLP_HEADERS="authorization=<MW_API_KEY>"

export OTEL_TRACES_EXPORTER=otlp
export OTEL_METRICS_EXPORTER=otlp
export OTEL_LOGS_EXPORTER=otlp

export OTEL_SERVICE_NAME=my-service

This configuration uses OTLP/gRPC to export telemetry directly to Middleware.

If your application uses OTLP/HTTP, change the protocol:

export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

The SDK uses the appropriate signal-specific paths when configured with the base endpoint.

For applications requiring separate HTTP endpoints, configure:

export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://<MW_UID>.middleware.io:443/v1/traces
export OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=https://<MW_UID>.middleware.io:443/v1/metrics
export OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=https://<MW_UID>.middleware.io:443/v1/logs

Application instrumentation#

If your application is not already instrumented, configure an OpenTelemetry SDK before exporting telemetry.

Middleware supports standard OpenTelemetry instrumentation across multiple languages.

Use automatic instrumentation where supported, and manual instrumentation for custom spans and metrics.

Node.js#

Install the OpenTelemetry SDK, auto-instrumentation package, and OTLP exporters:

npm install \
  @opentelemetry/sdk-node \
  @opentelemetry/auto-instrumentations-node \
  @opentelemetry/exporter-trace-otlp-grpc \
  @opentelemetry/exporter-metrics-otlp-grpc

{Retain the existing Node.js otel.js initialization and custom span code blocks.}

Python#

Install the OpenTelemetry distribution and exporter:

pip install opentelemetry-distro opentelemetry-exporter-otlp

opentelemetry-bootstrap -a install

Run the application with auto-instrumentation:

export OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true
export OTEL_PYTHON_LOG_CORRELATION=true

opentelemetry-instrument \
  --traces_exporter otlp \
  --metrics_exporter otlp \
  --logs_exporter otlp \
  --service_name my-service \
  python app.py

Java#

Use the OpenTelemetry Java Agent or SDK to instrument your application.

{Retain the existing Java Gradle dependencies and Java Agent execution code blocks.}

.NET#

Install the OpenTelemetry exporter and instrumentation packages:

dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Instrumentation.Http
dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol

{Retain the existing .NET tracing, metrics, and logging configuration code blocks.}

Advanced configuration#

Configure sampling#

Sampling controls the amount of trace data collected and exported.

Head-based sampling#

Configure sampling at the application SDK level.

For example, to sample approximately 20% of root traces:

export OTEL_TRACES_SAMPLER=parentbased_traceidratio
export OTEL_TRACES_SAMPLER_ARG=0.2

Child spans follow the parent's sampling decision.

Tail-based sampling#

Tail-based sampling is configured in the Collector after trace data is received.

Use it when you want to retain traces based on conditions such as error status or latency.

Add the tail_sampling processor to your Collector configuration.

{Retain the existing tail-sampling configuration example, updating the exporter endpoint to use the OTLP/HTTP base URL.}

Note: Tail sampling requires a Collector distribution that includes the tail_sampling processor. Configure the batch processor after sampling.

Configure context propagation#

Context propagation connects spans across services and enables correlation between frontend sessions, backend traces, and logs.

Use a consistent propagation format across applications.

For standard OpenTelemetry environments:

export OTEL_PROPAGATORS=tracecontext,baggage

For applications using Middleware RUM with B3 propagation:

export OTEL_PROPAGATORS=b3

Make sure your frontend, backend, and API gateway use compatible propagation headers.

To correlate logs with traces, include trace_id and span_id in application log records.

Configure resource attributes#

Use OpenTelemetry resource attributes to identify services and environments.

export OTEL_RESOURCE_ATTRIBUTES="service.name=checkout-service,service.version=1.2.3,deployment.environment=production"

Common attributes include:

Use consistent resource attributes across applications to make telemetry easier to filter and correlate.

Configure custom metrics#

Use OpenTelemetry metrics APIs to record application-specific measurements such as request counts, processing duration, and queue sizes.

Supported instruments include counters, histograms, and gauges.

{Retain the existing Node.js, Python, and Java custom metric examples from the source documentation.}

Use consistent metric names, appropriate units, and low-cardinality attributes.

Troubleshooting#

No telemetry in Middleware#

If the Collector is running but no data appears:

• Verify that the Collector can connect to your Middleware target URL over HTTPS on port 443.
• Confirm that the authorization header contains a valid Middleware API key.
• Check that the exporter protocol matches the configured endpoint.
• Verify that applications can reach the Collector on port 4317 or 4318.
• Check the Collector logs for exporter or authentication errors.

OTLP exporter connection errors#

If the Collector reports connection failures:

• Verify that the Middleware target URL is correct.
• Confirm that outbound HTTPS traffic is allowed.
• Check DNS resolution and TLS connectivity.
• Make sure OTLP/gRPC uses the root endpoint and OTLP/HTTP uses the correct signal paths.

Application traces are missing#

If metrics or logs appear but application traces are missing:

• Verify that tracing is enabled in the application SDK.
• Confirm that the application is generating requests.
• Check that OTEL_TRACES_EXPORTER is set to otlp, if required by the SDK.
• Verify that the Collector traces pipeline includes the OTLP receiver and Middleware exporter.

Trace correlation is not working#

If traces are not connected across services:

• Verify that the same context propagation format is used across services.
• Check that API gateways and proxies forward the required propagation headers.
• Confirm that application logs contain trace and span identifiers.

High telemetry volume#

If the amount of collected telemetry is higher than expected:

• Configure head-based sampling in application SDKs.
• Use tail-based sampling in the Collector to retain selected traces.
• Apply filtering or attribute processors where appropriate.
• Review the Collector's batching and memory configuration.

If the issue persists, contact Middleware Support at support@middleware.io.