Node.js

TracesMetricsApp LogsCustom LogsProfiling

This guide provides instructions to set up Application Performance Monitoring (APM) in a Node.js application. You can also find these instructions on the installation page in your Middleware account. For example code, view here.

Prerequisites#

  1. Node.js 18.17.1+: Verify with node --version
  2. Python Version 3.8+ Verify with python3 --version

Installation#

1 Install Node.js APM Package#

Run the following command in your terminal:

1npm install @middleware.io/node-apm --save

2 Setup Options#

Host based APM requires the Middleware Agent which can be installed using the relevant OS based instructions. here.

Method 1: Direct Code Initialization#

Add the following lines to the beginning of your application code base. The access token is your account key, which can be found on the Installation page.

If you want to track APM data across different deployments of your application. You can pass along the app.version,

For Example, If your deployment version is 1.2.0 the config should look something like this.

Method 2: Add the tracker with command line arguments#

Create a file named instrument.js or instrument.ts in your project root:

To ensure the tracker is initialized before any other code runs, use the --require flag when starting your Node.js application:

Method 1: Direct Code Initialization#

Add the following lines to the beginning of your application code base. The access token is your account key, which can be found on the Installation page.

If you want to track APM data across different deployments of your application. You can pass along the app.version,

For Example, If your deployment version is 1.2.0 the config should look something like this.

Method 2: Add the tracker with command line arguments#

Create a file named instrument.js or instrument.ts in your project root:

To ensure the tracker is initialized before any other code runs, use the --require flag when starting your Node.js application:

3 Container Variables#

No Container Variables Required in case of serverless mode

Docker#

Applications running in a container require an additional environment variable. If your application is not running in a container, move to Step 4.

For Docker containers, add the following environment variable to your application.

The DOCKER_BRIDGE_GATEWAY_ADDRESS is the IP address of the gateway between the Docker host and bridge network. This is 172.17.0.1 by default. Learn more about Docker bridge networking here

Kubernetes#

Then add the following environment variable to your application deployment YAML file.

4 Capture Application Data#

Traces#

Distributed tracing is automatically enabled upon completion of Step 2.

Span Attributes#

For any automatically instrumented endpoints, add the following code snippet:

1router.post("/email/send", async (req, res) => {
2  const email  = req.body.email;
3  tracker.setAttribute("email", email);
4  res.json({ status: "email sent" });
5});

Custom Logs#

To ingest custom logs into Middleware, utilize the following functions inside your logging method based on desired log severity levels.

To add stack traces along with the error log, use the following error tracking function.

Custom Metrics and Spans#

Create custom instruments and spans by exposing the meter and tracer with the following pattern:

Profiling#

Application Profiling is auto-configured upon completing Step 2.

Stack Traces#

Use the errorRecord method to record a stack trace when an error occurs. See an example of this method below.

Continuous Profiling#

Continuous profiling captures real-time performance insights. Access this feature under the APM > Continuous Profiling section in the Middleware platform.

Continuous profiling adds CPU/wall-clock sampling overhead to your application. We recommend enabling it in non-production environments (dev, staging) first to evaluate the performance impact before turning it on in production.

When using the @middleware.io/node-apm package, continuous profiling is enabled by default. If you've explicitly disabled it, just set enableProfiling: true in your tracker.track() call to turn it back on.

Enabling Continuous Profiling on an Existing Node.js Deployment (Kubernetes)#

If your Node.js app is already running in Kubernetes and you want to enable Pyroscope-based continuous profiling without rebuilding or redeploying the app image, you can patch the existing Deployment to add an init container that installs the Pyroscope Node.js SDK into a shared volume, then preload it into the app container via NODE_OPTIONS.

How it works#

  • An init container (pyroscope-injector) installs @pyroscope/nodejs into an emptyDir volume shared with the app container, and writes a small loader script that calls Pyroscope.init() / Pyroscope.start().
  • The app container mounts that same volume and gets NODE_OPTIONS=--require=/pyroscope/pyroscope-loader.js, which preloads the profiler before the app's own entrypoint runs.
  • All of this is added via a kubectl patch using strategic merge, so it only adds to the existing volumes, initContainers, env, and volumeMounts lists — it does not remove or replace anything else already configured on the deployment.

Prerequisites#

  • kubectl pointed at the target cluster/context.
  • The name of the Deployment and Namespace you want to profile.
  • The name of the container inside that pod spec that runs the Node.js process.
  • Your Middleware project UID (used as both the profiling server subdomain and the tenant ID) — auto-filled below when you're logged in and select a project.

One-shot patch command#

Fill in the deployment-specific placeholders (<DEPLOYMENT_NAME>, <NAMESPACE>, <CONTAINER_NAME>, <APP_NAME>) and run. <MW_UID> is auto-populated with your Middleware project UID once you select a project below (while logged in) — otherwise replace it manually.

Placeholder reference
PlaceholderDescription
<DEPLOYMENT_NAME>Name of the target Deployment
<NAMESPACE>Namespace the deployment lives in
<CONTAINER_NAME>The container in the pod spec that runs node — must match exactly, since the merge keys off container name
<APP_NAME>Service name to show up in profiling data
<MW_UID>Your Middleware project UID; used for both PYROSCOPE_SERVER_ADDRESS and PYROSCOPE_TENANT_ID — auto-filled if you're logged in and select a project above

Why this is safe to apply directly#

volumes, initContainers, containers[].volumeMounts, and containers[].env are all strategic-merge lists keyed by name (or mountPath for volume mounts). That means this patch only adds entries — it won't remove or clobber the target container's other env vars, mounts, or init containers.

Patching spec.template changes the pod template hash, so Kubernetes automatically rolls the deployment. No separate kubectl rollout restart is required.

If the target container already sets NODE_OPTIONS, this patch will overwrite that entry rather than append to it (merge-by-name replaces the whole value). If that's the case, edit the patch so --require=/pyroscope/pyroscope-loader.js is appended to the existing NODE_OPTIONS value instead of replacing it.

Init container failures never block your app#

By default, Kubernetes treats init containers as blocking — if pyroscope-injector exited with a non-zero status (for example, if npm install failed because of no network access or a node-gyp build issue), the pod would get stuck in Init:CrashLoopBackOff and the main app container would never start. The script in this patch is written to avoid that:

  • Every install step (apt-get, npm init, npm install) is suffixed with || true, and the script doesn't use set -e, so a failed install doesn't abort the script.
  • pyroscope-loader.js is always written, and the script always ends with exit 0 — so the init container always succeeds and the pod always proceeds to start the app container, whether or not the profiler was installed.
  • The loader script itself wraps Pyroscope.init()/Pyroscope.start() in a try/catch. If the module failed to install, or profiler startup throws for any other reason, it logs [pyroscope] profiler failed to start, continuing without profiling and lets --require return normally instead of crashing the app process.

Worst case if profiling can't be enabled: you lose profiling data and see that log line — the app itself starts and runs exactly as it would without this patch.

Verifying the rollout#

You should see [pyroscope] profiler started in the app container logs, along with the configured application name and endpoint. If profiling failed to install or start, you'll instead see [pyroscope] profiler failed to start, continuing without profiling: <reason> — the app container will still be Running as usual; only profiling data will be missing.

Environment Variables#

The following environment variables can be used to configure the Middleware APM tracker:

VariableDescriptionPossible Values
MW_TARGETSets the target URLYour Middleware URL
MW_API_KEYSets the access tokenYour Middleware API key
MW_PROJECT_NAMESets the project nameAny string
MW_SERVICE_NAMESets the service nameAny string
MW_APM_TRACES_ENABLEDEnables/disables trace collection"true" or "false"
MW_APM_METRICS_ENABLEDEnables/disables metric collection"true" or "false"
MW_CONSOLE_EXPORTEREnables/disables console exporter"true" or "false"
MW_AGENT_SERVICESets the host for the APM service when using MW AgentValid hostname or IP address
OTEL_NODE_RESOURCE_DETECTORSSpecifies which resource detectors to useComma-separated list (see below)
MW_NODE_DISABLED_INSTRUMENTATIONSSpecifies which instrumentations to disableComma-separated list (see below)

Customizing Resource Detection with OTEL_NODE_RESOURCE_DETECTORS#

Specifies which resource detectors to use. Each detector provides specific information about the application's environment.

Default: env, process, host, os, container, serviceinstance

Possible values: env, process, serviceinstance, os, host, container, aws, azure, gcp, none, all

Example: OTEL_NODE_RESOURCE_DETECTORS=env,process,host,aws

Disabling Specific Instrumentations with MW_NODE_DISABLED_INSTRUMENTATIONS#

Disables specific instrumentations to exclude certain types of data collection.

Note: File System (fs) instrumentation is permanently disabled due to performance issues.

Possible values: dns, net

Example: MW_NODE_DISABLED_INSTRUMENTATIONS=dns,net

Available instrumentations may vary depending on your Middleware APM version and configuration.

Configuration Options#

In addition to using environment variables, you can configure the Middleware APM tracker directly through the tracker.track() call. Here are the configuration options you can use: (env variables have higher priority)

OptionTypeDefaultDescription
targetstringN/ASets the target URL for the APM service
accessTokenstringN/ASets the access token for authentication
projectNamestringN/ASets the project name
serviceNamestringN/ASets the service name
pauseTracesbooleanfalseDisables trace collection
pauseMetricsbooleanfalseDisables metric collection
consoleExporterbooleanfalseEnables console exporter
customResourceAttributesobjectN/ASets custom resource attributes
disabledInstrumentationsstringN/ASpecifies which instrumentations to disable (comma-separated)
consoleLogbooleanfalseEnables consoleLog logs collection
consoleErrorbooleanfalseEnables consoleError logs collection
enableSelfInstrumentationbooleanfalseEnables self-instrumentation for the profiling traces
enableProfilingbooleantrueEnables profiling functionality for performance analysis
excludeHttpTracesobjectN/AExclude specific HTTP requests from tracing (see below)

Usage Example#

Excluding HTTP Requests from Tracing#

You can exclude specific HTTP requests from tracing using the excludeHttpTraces configuration. This allows you to control which incoming and outgoing HTTP requests are traced.

Programmatic Configuration#

1const tracker = require('@middleware.io/node-apm');
2tracker.track({
3  // ...other config...
4  excludeHttpTraces: {
5    incoming: {
6      urls: ['/health', '/metrics', '/status'],
7      methods: ['OPTIONS', 'HEAD']
8    },
9    outgoing: {
10      urls: ['/internal-api', '/telemetry', '/profiling'],
11      methods: ['TRACE']
12    }
13  }
14});

Environment Variables#

  • Exclude incoming requests:

    • MW_EXCLUDE_INCOMING_HTTP_METHODS="OPTIONS,HEAD,TRACE"
    • MW_EXCLUDE_INCOMING_HTTP_URLS="/health,/metrics,/status"
  • Exclude outgoing requests:

    • MW_EXCLUDE_OUTGOING_HTTP_METHODS="TRACE"
    • MW_EXCLUDE_OUTGOING_HTTP_URLS="/internal-api,/telemetry,/profiling"

Use Cases#

  • Exclude health check endpoints from tracing:
    • excludeHttpTraces: { incoming: { urls: ['/health', '/ready', '/live'] } }
    • Or: MW_EXCLUDE_INCOMING_HTTP_URLS="/health,/ready,/live"

Graceful Shutdown (Optional)#

The Middleware APM tracker provides a sdkShutdown function for graceful shutdown. This ensures all pending telemetry data is sent before your application exits. Call tracker.sdkShutdown() when your application is about to exit to ensure all data is properly flushed.

Usage#

1const tracker = require('@middleware.io/node-apm');
2
3// Initialize the tracker
4tracker.track({
5  // Your configuration options
6});
7
8// Handle application shutdown
9process.on('SIGINT', async () => {
10  console.log('SIGINT signal received. Shutting down gracefully');
11  await tracker.sdkShutdown();
12  // Perform other cleanup tasks
13  process.exit(0);
14});

Legacy Node.js Support (Node.js 10+)#

For applications running on legacy Node.js versions (10.x and above), Middleware provides a separate package with limited feature support and uses older OpenTelemetry libraries.

FeatureStatusNotes
TracesAuto-instrumentation supported
MetricsManual instrumentation only
ProfilingNot supported
LogsBasic support

Installation#

Install the legacy package using npm:

1npm install @middleware.io/node-apm-legacy

Basic Setup#

Add the following code at the entry point of your application:

1const tracker = require('@middleware.io/node-apm-legacy');
2tracker.track({
3    serviceName: "your-service-name",
4    accessToken: "<MW_API_KEY>",
5    target: "https://<MW_UID>.middleware.io:443",
6});
7// must come before importing any other module.
8// At the very top of the entrypoint
9const express = require('express'); // other dependencies
1const tracker = require('@middleware.io/node-apm-legacy');
2tracker.track({
3    serviceName: "your-service-name",
4    accessToken: "<MW_API_KEY>",
5});
6// must come before importing any other module.
7// At the very top of the entrypoint
8const express = require('express'); // other dependencies

Basic Troubleshooting Guidelines#

If you're experiencing issues with the Middleware APM tracker or not seeing the expected data, try the following steps:

  1. Enable Debug Logging: Set the DEBUG flag to true in your configuration or use the environment variable to get more detailed logs:

    1tracker.track({
    2  // other options...
    3  DEBUG: true
    4});
  2. Check Configuration: Ensure all required fields (like serviceName , accessToken , target and accessToken) are correctly set.

  3. Initialize Tracking Function: Make sure your tracker.track() is initialized at the very top of the main server file.

  4. Instrumentation Issues: If specific instrumentations aren't working, check if they're accidentally disabled in your configuration.

  5. Resource Detection: If you're not seeing expected resource attributes, verify your OTEL_NODE_RESOURCE_DETECTORS setting.

  6. Restart Your Application: Sometimes, a simple restart after configuration changes can resolve issues.

  7. Agent Health Check: If you're using the Middleware Agent, the SDK performs a health check at startup. If you see a warning message about the health check failing, it could be due to:

    • Incorrect value of MW_AGENT_SERVICE

    If the problem persists, verify your agent configuration and ensure it's running correctly.

  8. OpenTelemetry Dependency Conflicts: If you're using OpenTelemetry in your application code alongside the Middleware Node SDK, ensure that the versions are compatible. Conflicting versions can lead to unexpected behavior. Check the OpenTelemetry version used by the Middleware SDK (you can find this in the SDK's package.json) and make sure your application's OpenTelemetry dependencies match or are compatible with this version.

If problems persist after trying these steps, please contact Middleware support with your debug logs and configuration details (with sensitive information redacted).

Frequently Asked Questions (FAQ)#

node-gyp Failing or Missing Dependencies

If node-gyp fails or dependencies are missing, run the following commands:

1sudo apt-get build-dep build-essential
2sudo apt-get install gcc g++ make

Running Apps on Docker

If you're unable to run your Node.js app on Docker, ensure port 9319 is open to allow proper firewall configurations.

Installation successful but no data appears / Existing OpenTelemetry dependency older version issue

If you're seeing the following error, especially when DEBUG: true:

Error: @opentelemetry/api: Registration of version vx.x.x for trace does not match previously registered API vy.y.y

This error occurs due to a version conflict in OpenTelemetry dependencies:

  • Your project (or one of its dependencies) is using OpenTelemetry API older version
  • Our package requires OpenTelemetry API vx.x.x

This version mismatch prevents proper registration of the OpenTelemetry API, which blocks data collection.

Override the older OpenTelemetry API version by explicitly installing vx.x.x:

1npm i @opentelemetry/api@x.x.x --save

Docker Image Failing

If your Docker image build is failing, it might be due to missing build dependencies. Add the following lines to your Dockerfile:

1RUN apk --no-cache add build-base
2RUN apk add --update --no-cache python3 && ln -sf python3 /usr/bin/python

These commands install necessary build tools and Python, which are often required for compiling native addons.

ESM Modules Auto-instrumentation not working

If your JavaScript or TypeScript code ( compiled to ES6 Modules (ESM)) and auto-instrumentation isn't working for few libraries.

  1. Install the required package
1npm i @opentelemetry/instrumentation@0.54.1 // or any other new version
  1. Use this startup command:
1node --experimental-loader=@opentelemetry/instrumentation/hook.mjs --import instrument.js app.js

Pyroscope profiling is not available

This warning appears because Pyroscope is an optional dependency that requires system-level build tools. It may fail if:

  • Required system dependencies are missing
  • node-gyp build fails
  • Running in Docker without build essentials
  • Not supported for few machines

Solution For local development, install build dependencies:

1sudo apt-get build-dep build-essential python3
2sudo apt-get install gcc g++ make

For Docker, add build tools in your Dockerfile:

1RUN apk --no-cache add build-base
2RUN apk add --update --no-cache python3 && ln -sf python3 /usr/bin/python

Memory Spike & Duplicate Instrumentation with --require / --import Flag

You see instrumentation loading multiple times in logs and high memory usage (DEBUG should be true) :

1Loading instrumentation for @opentelemetry/instrumentation-express
2...
3[INFO] API server running
4Loading instrumentation for @opentelemetry/instrumentation-express  #Loading again!
5...

This happens when using node --require or node --import flags with node services that create worker threads or child processes:

  • pino with transports (pino-pretty)
  • Cluster mode
  • Bull/BullMQ queues
  • Any library using worker_threads

Root Cause: Node.js applies --require/--import to every process, including workers, causing multiple instrumentation initializations.

Impact: 2-3x memory usage (~150-300MB instead of ~75-100MB) and random issues

Avoid:

1{
2  "scripts": {
3    "start": "node --require ./instrumentation.js ./index.js"
4  }
5}

Use instead:

1// index.ts - Import instrumentation FIRST
2import './instrumentation';
3
4import express from 'express';
5// ... rest of imports

Or CommonJS:

1// index.js
2require('./instrumentation');
3
4const express = require('express');
5// ... rest of code

After fix, instrumentation should load only once in debug logs and issue should be fixed.

Need assistance or want to learn more about Middleware? Get in touch with us via our Contact Us or join our Slack channel.