Installing the Datadog Agent for Middleware

This guide explains how to configure the Datadog Agent to send data exclusively to Middleware. Middleware supports the ingestion of APM traces, metrics and logs from the Datadog Agent

This feature is supported in Datadog's Linux and Kubernetes Agent

Prerequisites

1 Traces

Datadog Agent version 6.7.0 or above

2 HTTP Logs

Datadog Agent version 6.13 or above

3 Metrics

Datadog Agent version 6.17 or above

Datadog Linux Agent

Datadog's Linux agent stores configuration in YAML format. This file is located in /etc/datadog-agent/datadog.yaml by default. You will need to update this file to enable sending metrics, traces, and logs to the Middleware platform.

Initial Configuration

Add the following configuration to your datadog.yaml file:

api_key: "<MW_API_KEY>"
dd_url: https://<MW_UID>.middleware.io:443

Trace Configuration

Add the following apm_dd_url within the apm_config section in your datadog.yaml file:

If you already have the apm_config section in your datadog.yaml file, add an apm_dd_url endpoint in the same section. Datadog agent does not support multiple apm_config sections.

The Linux Datadog Agent is configured by default in the following YAML file: /etc/datadog-agent/datadog.yaml

apm_config:
  enabled: true
  apm_dd_url: https://<MW_UID>.middleware.io:443

Log Configuration

Add the following logs_dd_url within the logs_config section in your datadog.yaml file:

logs_enabled: true
logs_config:
  logs_dd_url: https://<MW_UID>.middleware.io:443
  container_collect_all: true
  logs_no_ssl: true
  use_http: true
  auto_multi_line_detection: true

Configuration

api_key: <MW_API_KEY>
dd_url: https://<MW_UID>.middleware.io:443
apm_config:
  enabled: true
  apm_dd_url: https://<MW_UID>.middleware.io:443
logs_enabled: true
logs_config:
  logs_dd_url: https://<MW_UID>.middleware.io:443
  container_collect_all: true
  logs_no_ssl: true

In order to get visibility into the processes running on your infrastructure, enable live processes collection under the process_config section as shown below:

process_config:
  process_collection:
    enabled: true

Restart Datadog Agent

Run the following code in your terminal to restart the Datadog Agent:

sudo systemctl restart datadog-agent

Replace <MW_API_KEY> with your Middleware API Key and {ACCOUNT-UID} with your Middleware account UID.

Datadog Kubernetes Agent

Datadog Kubernetes agent supports Helm and Operator modes of installation. Middleware can ingest traces, metrics, and logs from the Datadog Kubernetes agent installed using any of these modes.

Helm Chart Installation

If you have installed the Datadog Agent using their Helm chart, you must add the following sections to your datadog-values.yaml.

Replace <your-cluster-name> with your actual cluster name, and login to see your account's MW_API_KEY and MW_UID.

datadog:
  dd_url: https://<MW_UID>.middleware.io:443
  logsEnabled: true
  logs:
    enabled: true
    containerCollectAll: true
    containerCollectUsingFiles: true
    autoMultiLineDetection: true
  clusterName: <your-cluster-name>
  env:

You can remove the logs_config and/or apm_config sections if you don't want to send logs or APM to Middleware.

You can also remove the orchestrator section or other sections to prevent sending Kubernetes metrics and metadata info.

Please provide accurate ClusterName values where required in the YAML, as it helps in correctly correlating data in Middleware. Any issues with cluster names can lead to inconsistencies.

FAQs

How do I troubleshoot if I can't see my data in Middleware?

For Datadog Linux agent, ensure that the agent is in an active state. You can issue the following command to find out the status:
```
sudo systemctl status datadog-agent
```
Verify that the Middleware API key and target URLs are correct.
Ensure there's only one apm_config and logs_config section in the configuration file (/etc/datadog/datadog.yaml).
For Datadog Kubernetes agent, ensure that the datadog-agent and cluster-agent pods are operational. You should check that liveness and readiness checks are reporting healthy for both of these pods.
For Kubernetes Helm chart installation method, check the datadog-values.yaml and ensure that the agents section is at the same level as the datadog section. The agents section should NOT be inside the datadog section.

Need assistance or want to learn more about Middleware? Contact our support team in Slack.