---
title: "Client-side Evaluations (SDK)"
description: "Run LLM evaluations in your application with the middleware-llmobs Python SDK: submit computed scores, run an LLM judge, or write a custom evaluator."
section: "LLM Observability"
breadcrumb: "LLM Observability > Evaluations > Client-side (SDK)"
canonical: "https://docs.middleware.io/features/llm/evaluations/client-side"
---
# Client-side Evaluations (SDK)

Client-side evaluations run in your application code using the [`middleware-llmobs`](/features/llm/middleware-sdk) SDK. You define how responses are evaluated, whether by submitting a computed value, running your own LLM-as-a-judge, or executing a deterministic check. The SDK then attaches the result to the trace span and exports it to Middleware via OTLP.

Use client-side evaluations when you need logic that lives in your app: RAG-aware checks, offline or CI evaluation, deterministic checks (regex, JSON validity, similarity thresholds), or a judge you fully control.

> **Note:** For no-code, always-on evaluation of production traffic, use [server-side evaluations](/features/llm/evaluations/server-side) instead. Middleware runs the judge for you.

## Score value types

An evaluation result can use one of three value types, the same types used everywhere in Middleware evaluations:

- **boolean** — a pass/fail result (e.g. a content-safety check).
- **score** — a number, with a passing range you define.
- **categorical** — a label from a defined set.

Each can carry an optional **pass/fail assessment** and the **reasoning** behind it.

## Three ways to evaluate

### 1. Submit a precomputed result

If your application has already computed an evaluation result, record it with `submit_evaluation`. By default it binds to the active span:

```python
from middleware.llmobs import submit_evaluation

submit_evaluation(
    label="response_not_empty",   # ^[a-zA-Z][a-zA-Z0-9_]*$
    value=True,                   # bool -> boolean, int/float -> score, str -> categorical
    assessment="pass",            # optional: "pass" | "fail"
    reasoning="The model returned a non-empty answer.",
)
```

### 2. Run an LLM-as-a-judge in code

Use `LLMJudge` to evaluate model outputs with another model and submit the result in a single step with `evaluate_and_submit`. Because the SDK never imports a provider, you supply a thin client adapter for your model.

### 3. Write a custom (non-LLM) evaluator

For deterministic checks — regex, JSON validity, length, similarity thresholds — use the `@evaluator` decorator for quick functions, or subclass `BaseEvaluator` for stateful/configurable evaluators.

## Full reference

The SDK guide contains the complete client-side evaluation API, including:

- `submit_evaluation`
- `LLMJudge` and `AsyncLLMJudge`
- Structured output types
- `@evaluator` and `BaseEvaluator`
- `EvaluatorContext` and `EvaluatorResult`
- Flushing support for short-lived processes

→ [Middleware SDK for Python — Evaluations](/features/llm/middleware-sdk#6-evaluations)

## Where results appear

Submitted evaluations are attached to the span they evaluate and appear in the span’s **Evaluation** tab in **LLM Traces**, just like server-side evaluation results.

The SDK exports evaluation results using the same schema as server-side evaluations, following the OpenTelemetry [GenAI semantic conventions](https://github.com/open-telemetry/semantic-conventions-genai). Each evaluation is exported as:

- An OTLP **log record** containing the result and `gen_ai.evaluation.*` attributes, correlated to the evaluated span through `trace_id` and `span_id`.
- Gauge **metrics** under `gen_ai.evaluations.*` for dashboards, pass-rate tracking, and alerting.

Evaluation data is exported over OTLP/HTTP. For the exact attributes and metric names, see [Data exported](/features/llm/evaluations/server-side#data-exported) on the server-side page.

> **Note:** Need help? Get in touch with us via our [Contact Us](https://middleware.io/contact-us/) or join our [Slack community](https://middlewarepublic.slack.com/join/shared_invite/zt-39l76rpo6-ZNGfpA_4U_kYL6oTuYAgmQ).