Setting up OpenLIT SDK for Middleware (Python)

This guide will walk you through the process of setting up the OpenLIT SDK to work with Middleware for LLM Observability in a Python environment.

1. Install the SDK

Run the following command in your terminal:

pip install openlit

2. Initialize the SDK

You can initialize the OpenLIT SDK using either function arguments or environment variables:

Setup using function arguments

In your LLM application, initialize the OpenLIT SDK:

import openlit

openlit.init(
    otlp_endpoint="https://<MW_UID>.middleware.io:443",
    application_name="YOUR_APPLICATION_NAME",
    otlp_headers={
        "Authorization": "<MW_API_KEY>",
        "X-Trace-Source": "openlit",
    },
)

Setup using Environment Variables

Set the following environment variables:

export OPENLIT_OTLP_ENDPOINT="your-initial-uid"
export OPENLIT_APPLICATION_NAME="YOUR_APPLICATION_NAME"
export OPENLIT_OTLP_HEADERS='{"Authorization": "your-initial-token", "X-Trace-Source": "openlit"}'

Then, in your Python code, simply call:

import openlit

openlit.init()

The SDK will automatically use the environment variables for configuration.

3. Use the SDK

Here's an example of how to use OpenLIT to monitor OpenAI usage:

from openai import OpenAI
import openlit

openlit.init(
    otlp_endpoint="https://<MW_UID>.middleware.io:443",
    application_name="YOUR_APPLICATION_NAME",
    otlp_headers={
        "Authorization": "<MW_API_KEY>",
        "X-Trace-Source": "openlit",
    },

Viewing Your Traces and Metrics

After setting up the Traceloop SDK with Middleware, you'll be able to view your LLM application traces in your Middleware LLM Observability Section.

This integration provides instant visibility into everything happening within your LLM, including performance metrics and usage patterns.