TrueFoundry

What is Truefoundary?#

TrueFoundry AI Gateway is the proxy layer that sits between your applications and the LLM providers and MCP Servers. It is an enterprise-grade platform that enables users to access 1000+ LLMs using a unified interface while taking care of observability and governance.

Prerequisites#

Before integrating Middleware with TrueFoundry, ensure you have:

  1. TrueFoundry Account: Create a Truefoundry account and follow the instructions in our Gateway Quick Start Guide
  2. Middleware access: Log in to Middleware with your work email (the account your organization uses for Middleware).
  3. Middleware API key: Obtain the API key Middleware issued for OTLP trace ingestion (typically from Middleware project or organization settings — use the credential they designate for traces).
Kinesis Stream - Auto Method

Integration Steps#

TrueFoundry AI Gateway supports exporting OpenTelemetry traces to Middleware using your tenant OTLP ingest URL.

1 Collect your Middleware API key#

  1. Log in to Middleware with your work email and open your API key or organization settings (your admin may point you to the credential used for trace ingestion).
  2. Copy the secret and store it in a credential manager until you paste it into TrueFoundry gateway settings — you typically cannot retrieve the full secret again once generated.

If you are unsure which key fits this integration, confirm with Middleware support — this flow expects the key mapped to OTLP ingestion at https://<your-domain>.middleware.io:443/v1/traces (replace <your-domain> with your Middleware hostname prefix).

2 Configure OTEL Export in TrueFoundry#

  1. In the TrueFoundry dashboard, go to AI Engineering → Settings → OTEL Config (under Organisation, in the AI Gateway section).
    Kinesis Stream - Auto Method
  2. Click edit on the OTEL Config section to open the exporter form (if it is not already open).
  3. Enable the OTEL Traces Exporter Configuration toggle.
  4. Select HTTP Configuration.
  5. Enter the Middleware traces endpoint: https://<your-domain>.middleware.io:443/v1/traces
  6. Set Encoding to Proto.
Kinesis Stream - Auto Method

3 Configure Headers#

Middleware expects authentication through the Authorization header only. Paste the raw API key as the header value (do not add a Bearer prefix unless Middleware explicitly instructed you otherwise).

HeaderValue
Authorization<YOUR_MIDDLEWARE_API_KEY>

Click Save to apply your configuration.

4 Verify the Integration#

  1. Make some requests through the TrueFoundry AI Gateway.
  2. Navigate to the Monitor section in TrueFoundry to verify traces are being generated.
  3. Open Middleware and drill into traces for your project or environment to confirm spans sourced from TrueFoundry are appearing.
Kinesis Stream - Auto Method

Configuration Options#

Middleware Endpoint#

Middleware exposes ingest configuration similar to the following (use your tenant hostname):

ConfigurationValue
Traces Endpointhttps://<your-domain>.middleware.io:443/v1/traces
ProtocolHTTP
EncodingProto
AuthenticationAuthorization set to your Middleware API key (no extra headers required)

Need assistance or want to learn more about Middleware? Contact our support team at [email protected] or join our Slack channel.