June 17, 2026

Middleware v2.7.6 expands LLM observability with request details, a playground, evaluations, and GPU monitoring. The release also adds Amazon S3 telemetry routing, Kafka monitoring on Kubernetes, and OpsAI actions.

LLM Observability and GPU Monitoring#

  • Request details: Inspect token usage, cost, model, spans, stack traces, chat input, and related logs when investigating an LLM request.
LLM Observability: Updated LLM traces screen with more data such as total token per request, cost, model, span info…
  • Playground: Try models, output schemas, and tools with sample prompts to inspect responses.
LLM Observability: Playground Use different model, output schemas, add tool and see how LLM will give output for sample…
  • LLM-as-a-judge evaluations: Select an evaluator model, scope, rules, and acceptance criteria to assess response quality.
LLM Observability: LLM as a judge (Evaluation) Use a language model as the evaluator and define your scope, model and…
  • GPU monitoring: Review host and device utilization, occupancy, memory, temperature, power, bandwidth, and processes to investigate resource bottlenecks.
LLM Observability: GPU Monitoring Monitor GPU hosts and devices with its metrics such as utilizations, occupancy…
  • Dashboards: Added 50+ graphs covering tokens, cost, error rate, duration, LLM calls, evaluations, latency, and tool metrics.

Linux Setup and Cloud Integrations#

  • Linux setup: Configure detected databases and integrations directly from the Linux Instrumentation screen.
  • Kafka on Kubernetes: Added integration support for monitoring Kafka in Kubernetes environments.
  • AWS RDS: Added support for eight additional regions, including regions in Asia.

Telemetry Routing to Amazon S3#

Route metrics, logs, and traces to your own Amazon S3 bucket. Configure your AWS account ID, bucket name, and region under Settings → S3 Routing.

S3 Routing: Introduced a self-service S3 Routing configuration page under Settings → S3 Routing, allowing users to…

Log Anomaly Investigation#

Inspect log anomalies such as count spikes, behavior drift, and inactive patterns. Open an anomaly to review log properties, related logs, and AI analysis when investigating changes in logging behavior.

Anomaly in Log monitoring: You can see anomaly in logs by its patterns and different anomaly types such as count spike…
Anomaly in Log monitoring: We can see the details for each anomaly with log properties, related logs and analysis by AI.

OpsAI Investigation and Chat Actions#

  • Kubernetes commands: Run commands directly from an investigation or chat.
OpsAI: Run Kubernetes commands within the investigation/chat directly with one click.
  • Dashboard creation: Describe a dashboard in chat and receive a link to the generated dashboard.
  • Vercel issues: Investigate Vercel logs and fatal errors from the issue list and prepare pull requests for proposed fixes.
  • Synthetic tests: Ask questions about synthetic tests or create new tests from New Chat.

Middleware MCP Access and Loading Performance#

  • MCP access: Review and revoke third-party application access to Middleware data from the MCP Server access page.
Improvements: Added an MCP Server access page, allowing users to view and revoke third-party applications that have…
  • Loading performance: Improved APM, MongoDB, and OpsAI list and detail screens when working with large datasets.

Need assistance or want to learn more about Middleware? Get in touch with us via our Contact Us or join our Slack channel.

What did you think of this content?