Middleware LLM SDK - Overview

middleware-llmobs is Middleware’s first-party LLM Observability SDK, built on OpenInference and OpenTelemetry. Unlike the third-party Traceloop and OpenLIT integrations, it’s maintained by Middleware and ships both tracing and evaluations in one package.

What it gives you#

• One-line tracing — register(auto_instrument=True) configures OpenTelemetry and auto-instruments installed AI/ML libraries (OpenAI, Anthropic, LangChain, LlamaIndex, and many more) to produce OpenInference LLM spans.
• Evaluations — attach scores and verdicts to spans and traces with submit_evaluation, run LLM-as-judge checks with LLMJudge, or write custom evaluators — all exported as OTel logs and metrics.
• Standard OTel under the hood — configured with the usual OTEL_EXPORTER_OTLP_* environment variables and exported to Middleware over OTLP/HTTP.

Get started#

Python

Install the SDK, set up tracing, and add evaluations.

Cookbooks

Copy-paste recipes for tracing, agents, RAG, and evaluations.