Middleware LLM SDK - Cookbooks

End-to-end recipes for instrumenting real LLM applications with the Middleware LLM SDK. Each recipe is a focused, copy-paste example you can run against your own Middleware account.

New to the SDK? Start with the Setup guide to install middleware-llmobs, configure your endpoint, and call register(). Every recipe below assumes you’ve done that.

Getting started#

Trace an LLM application

Auto-instrument an OpenAI call and see your first trace in Middleware.

Trace an agent

Auto-instrument an Agno agent to trace its planning, tool calls, and LLM steps, then evaluate and monitor it.

Trace a RAG pipeline

Record the retrieval query and documents on a retrieval span with @retriever.

Track sessions & users

Group a multi-turn conversation and attribute traffic to a user.

Evaluations#

Evaluate with an LLM-as-judge

Score answer quality (e.g. toxicity) with a judge model and submit the verdict.

Write a custom evaluator

Run non-LLM checks (regex, similarity, thresholds) and submit the result.