Python

This guide walks you through setting up Application Performance Monitoring (APM) on a Python application. These instructions can also be found on the Installation page in your Middleware Account. View example code here.

Prerequisites

1 Python Version

Python 3 version 3.8 or above. Check your Python version with the following command:

Shell

python3 --version

2 Pip Version

pip version 23.1.2 or above. Check your pip version with the following command:

Shell

pip --version

Installation

Step 1: Install Python APM Package

Run the following command in your terminal:

Shell

$ pip install middleware-apm

Check if the middleware-apm has been installed with the following command:

Shell

$ pip list

Step 2: Initialize Middleware APM

Run the following command to generate the default middleware.ini config and bootstrap and install the required instrumentation packages (which internally uses opentelemetry-bootstrap --action=install):

Python3

$ middleware-apm init

Step 3: Import Middleware Tracker

Add the following lines to the entry point of your application, and as the first import:

Python3

import logging 
 
from middleware import MwTracker
tracker=MwTracker()

Step 4: Instrumentation Configuration

After initializing the APM using middleware-apm init, a default middleware.ini file is created at the root of your app directory.

This file contains the configuration for your application's instrumentation, such as traces, metrics, logs, and profiling.

You can modify the configuration directly inside the file or by using environment variables.

Sample Configuration File (middleware.ini)

The Common Attributes are ones we suggest always using, while the ones commented out are dependent on your use case.

# ------ Common Attributes ------
    service_name = python-apm-service

    access_token = "<MW_API_KEY>"

    collect_traces = true

    collect_metrics = true

    collect_logs = true

Available Configs and Env Variables

This table shows the available configuration settings from the above config file along with the corresponding environment variables that can be used instead.

All OpenTelemetry environment variables are supported and take the highest priority.

Step 5: Start Your Project

Run the following command to start your project with instrumentation:

Shell

middleware-apm run python app.py

Host Based Configuration

If you are deploying the host using Kubernetes or Docker you will need to set the mw_agent_service to the location of the mw-agent.

This variable can be ignored when the agent is running on the same host as the application.

Docker

Kubernetes

Add the mw_agent_service or export environment variable to your application:

Shell

docker run \
  -e MW_AGENT_SERVICE=<DOCKER_BRIDGE_GATEWAY_ADDRESS> \
  -p 8000:8000 -it --rm --name CONTAINER_NAME DOCKER_IMAGE:IMAGE_TAG

The DOCKER_BRIDGE_GATEWAY_ADDRESS is the IP address of the gateway between the Docker host and bridge network. This is 172.17.0.1 by default. Learn more about Docker bridge networking here

Add the following command to your Dockerfile after the pip install command:

Dockerfile

RUN middleware-bootstrap -a install

Identify the namespace where the Infra Agent is running:

Shell

kubectl get service --all-namespaces | grep mw-service

Then add the following environment variable to your application deployment YAML file:

Shell

MW_AGENT_SERVICE=mw-service.mw-agent-ns.svc.cluster.local

Serverless Configuration

If you are running your Python application in a serverless setup without the mw-agent, the MW_API_KEY and MW_TARGET variables are required when running the application:

MW_API_KEY='<MW_API_KEY>' MW_TARGET='https://<MW_UID>.middleware.io:443' middleware-apm run python app.py

Framework Specific Configurations

Specific frameworks require different configurations when being run, which can be found below:

DJANGO_SETTINGS_MODULE='mysite.settings' middleware-apm run python manage.py runserver

MW_API_KEY='<MW_API_KEY>' MW_TARGET='https://<MW_UID>.middleware.io:443' DJANGO_SETTINGS_MODULE='demo.settings' middleware-apm run gunicorn -c conf/gunicorn.conf.py --workers=4  --bind 0.0.0.0:8000 --timeout 120 demo.wsgi

MW_API_KEY='<MW_API_KEY>' MW_TARGET='https://<MW_UID>.middleware.io:443' middleware-apm run uvicorn main:app --host localhost --port 5002

Sending Custom Data

Custom Metrics

Create and use a meter to send custom metrics:

Shell

from opentelemetry.metrics import get_meter_provider

# Create a meter
meter = get_meter_provider().get_meter("custom_meter")

# Create a counter
request_counter = meter.create_counter(
    "request_counter", description="Counts the number of requests"
)

View more

Custom Traces

Create and use a tracer to send custom spans:

Shell

from opentelemetry.trace import get_tracer

# Create a tracer
tracer = get_tracer("custom_tracer")

# Start a span
with tracer.start_as_current_span("custom_span"):
    print("Doing some work within the span")

Custom Logs

Utilize Middleware's logging method to send logs with a given priority. It is recommended to integrate these calls function inside your existing logger:

Python3

logging.info("info sample")
logging.warning("Sample Warning Log") 
logging.error("Sample Error Log.", extra={'tester': 'Alex'})

Custom Attributes

All custom attributes collected will be available as filtering and grouping functions inside of the Middleware platform.

Using similar methods from the above Traces and Logs sections you can attach custom attributes to any trace, span or log:

Shell

from opentelemetry.trace import get_tracer

tracer = get_tracer("custom_tracer")

with tracer.start_as_current_span("span_with_attributes") as span:
    span.set_attribute("user.email", "[email protected]")
    span.set_attribute("user.id", 1234)

Adding Stack Traces

Use tracker.record_error() method to record a stack trace when an error occurs:

Python3

try:
    not_possible = 12/0
except ZeroDivisionError as e:
    tracker.record_error(e)

Continuous Profiling

By default the Python APM captures Continuous Profiling data which provides real-time performance insights from your application to enable rapid identification of resource allocation, bottlenecks, and more. Navigate to the Continuous Profiling section to learn more about using Continuous Profiling with Middleware.

Example application

Below is a sample python flask server application.

Shell

import logging
from middleware import MwTracker
tracker=MwTracker()

from flask import Flask
logging.getLogger().setLevel(logging.INFO)
logging.info("Application initiated successfully.", extra={'Tester': 'Alex'})

app = Flask(__name__)

View more

Start the server

MW_SERVICE_NAME="flask-server" middleware-apm run python app.py

Troubleshooting and Debugging

To help troubleshoot and debug issues with your application's telemetry, you can enable specific settings in the middleware.ini file or use environment variables. These settings allow you to export telemetry data to the console, save them to log files, or disable basic information logs.

#in the middleware.ini file
disable_info = true

#as an env variable
MW_DISABLE_INFO = true

#in the middleware.ini file
console_exporter = true

#as an env variable
MW_CONSOLE_EXPORTER = true

#in the middleware.ini file
debug_log_file = true

#as an env variable
MW_DEBUG_LOG_FILE = true

yum -y install python3-devel
yum -y install gcc-c++

apt install -y python3-dev
apt install -y build-essential

apk add python3-dev
apk add build-base