Installing the Middleware agent on Azure Virtual Machine Scale Set (VMSS)

The Middleware Infrastructure Agent monitors Linux virtual machines in Azure Virtual Machine Scale Sets (VMSS). Deployment through the Azure Custom Script Extension helps keep agent installation consistent as the scale set adds instances.

After the extension is added to the VMSS model, the installation script can be applied to existing instances and automatically runs on new instances created during scale-out.

Use this setup when you want to:

Install the Middleware Agent across an existing Linux VM scale set without manually connecting to each VM.
Keep Agent installation consistent across VMSS instances.
Automatically install the Agent on new instances created during scale-out.

How it works#

The Middleware Agent is installed on each VM using the Azure Custom Script Extension.

The installation process consists of the following steps:

1. Prepare the installation script#

Detects the Linux distribution.
Selects the appropriate package manager and installation method.
Waits for active package manager operations to complete.
Installs the Middleware Agent using your API key and target URL.

2. Attach the script to the VMSS#

Adds the Custom Script Extension to the VMSS model.
Configures the extension to execute the Middleware installation script.

3. Roll out the configuration#

Apply the updated VMSS model to existing instances.
New instances created after the extension is configured receive the extension automatically.

If an extension deployment fails or you change the underlying VM image, you may need to update, reimage, or recreate the affected instances to apply the current VMSS configuration.

Prerequisites#

Before installing the Middleware Infrastructure Agent, make sure you have the following:

Middleware

A Middleware account.
Your Middleware API key (MW_API_KEY).
Your Middleware target URL (MW_TARGET).

Azure

An existing Linux Azure Virtual Machine Scale Set (VMSS).
Azure CLI installed and authenticated.
Azure permissions to:
- Read and update the VMSS.
- Add, update, and remove VMSS extensions.

Supported distributions

This installation supports Linux distributions that use one of the following package managers: apt ,yum , dnf , zypper

Installation#

1 Prepare the installation script#

Create a file named install-middleware-agent.sh on the machine where you run Azure CLI.

bash

#!/bin/bash

# Detect OS and package manager

echo "Detecting operating system..."

if [ -f /etc/os-release ]; then
    . /etc/os-release
    OS=$ID
    echo "Detected OS: $OS"
else
    echo "ERROR: Cannot detect OS"
    exit 1
fi

# Determine package manager and installation script

if command -v apt-get >/dev/null 2>&1; then
    # Debian/Ubuntu
    PKG_MGR="apt"
    INSTALL_SCRIPT="https://install.middleware.io/scripts/deb-install.sh"
    LOCK_FILES="/var/lib/dpkg/lock-frontend /var/lib/apt/lists/lock /var/lib/dpkg/lock"
elif command -v yum >/dev/null 2>&1; then
    # RHEL/CentOS/Amazon Linux
    PKG_MGR="yum"
    INSTALL_SCRIPT="https://install.middleware.io/scripts/rpm-install.sh"
    LOCK_FILES="/var/run/yum.pid"
elif command -v dnf >/dev/null 2>&1; then
    # Fedora/RHEL 8+
    PKG_MGR="dnf"
    INSTALL_SCRIPT="https://install.middleware.io/scripts/rpm-install.sh"
    LOCK_FILES="/var/cache/dnf/metadata_lock.pid"
elif command -v zypper >/dev/null 2>&1; then
    # SUSE/openSUSE
    PKG_MGR="zypper"
    INSTALL_SCRIPT="https://install.middleware.io/scripts/rpm-install.sh"
    LOCK_FILES="/var/run/zypp.pid"
else
    echo "ERROR: Unsupported package manager"
    exit 1
fi

echo "Using package manager: $PKG_MGR"
echo "Installation script: $INSTALL_SCRIPT"

# Wait for package manager locks to be released

echo "Checking for package manager locks..."

timeout=300
elapsed=0

while true; do
    locked=false
    for lock_file in $LOCK_FILES; do
        if [ -f "$lock_file" ] && fuser "$lock_file" >/dev/null 2>&1; then
            locked=true
            break
        fi
    done

    if [ "$locked" = false ]; then
        break
    fi

    if [ $elapsed -ge $timeout ]; then
        echo "ERROR: Timeout waiting for package manager locks (waited ${timeout}s)"
        exit 1
    fi

    echo "$PKG_MGR is busy, waiting... (${elapsed}s elapsed)"
    sleep 10
    elapsed=$((elapsed + 10))
done

echo "Package manager available. Installing Middleware agent..."

# Install Middleware agent

MW_API_KEY="<MW_API_KEY>" \
MW_TARGET="https://<MW_UID>.middleware.io:443" \
bash -c "$(curl -L $INSTALL_SCRIPT)"

if [ $? -eq 0 ]; then
    echo "Middleware agent installed successfully"
    exit 0
else
    echo "ERROR: Middleware agent installation failed"
    exit 1
fi

Save the file and make it executable:

bash

chmod +x install-middleware-agent.sh

What this script does:

Detects the operating system using /etc/os-release.
Selects the appropriate package manager (apt, yum, dnf, or zypper) and Middleware installation script.
Waits for package manager locks to clear to avoid conflicts with ongoing system updates.
Runs the Middleware installer with MW_API_KEY and MW_TARGET set as environment variables.
Returns a non-zero exit code if the installation fails, allowing the Azure extension to report the failure.

2 Attach the Custom Script Extension to the VMSS#

From the directory containing install-middleware-agent.sh, run:

bash

az vmss extension set \
  --resource-group <your-resource-group> \
  --vmss-name <your-vmss-name> \
  --name CustomScript \
  --publisher Microsoft.Azure.Extensions \
  --version 2.1 \
  --protected-settings "{\"script\": \"$(base64 -w 0 install-middleware-agent.sh)\"}"

What this command does:

Adds the Custom Script Extension to the VMSS model.
Embeds the Base64-encoded install-middleware-agent.sh script in the extension configuration.
Ensures the script runs automatically when new VM instances are created.

Existing instances do not run the script automatically. You must update, reimage, or recreate them in the next step. If you run the command from a non-Linux environment, use the equivalent Base64 command that produces a single-line output without line breaks.

3 Roll out the agent to existing VM instances#

After attaching the extension to the VMSS model, roll out the change to existing instances.Choose the rollout strategy that best fits your environment.

Try this first. It refreshes the configuration of all instances and triggers the extension without recreating VMs.

bash

az vmss update-instances \
  --resource-group <your-resource-group> \
  --name <your-vmss-name> \
  --instance-ids "*"

Use this when:

You are fine with instances being updated in place
You want the change to apply across the scale set with minimal disruption

If this succeeds, the Custom Script Extension runs, the agent installs and you can move to verification.

Use a rolling upgrade when you want to apply the new extension configuration gradually across the scale set to reduce impact on production workloads.

bash

az vmss rolling-upgrade start \
  --resource-group <your-resource-group> \
  --name <your-vmss-name>

During a rolling upgrade, Azure:

Updates a subset of instances at a time
Waits for the instances to become healthy
Proceeds to the next batch

Even with a rolling upgrade, VMs are restarted. Any state that is not externalised will be lost on upgraded instances.

If you see errors like PropertyChangeNotAllowed or configuration drift, reimage the instances so they boot from the base image and apply the latest extension settings.

bash

az vmss reimage \
  --resource-group <your-resource-group> \
  --name <your-vmss-name> \
  --instance-ids "*"

This action:

Recreates the OS disk for each selected instance
Keeps the scale set configuration and networking
Causes the instance to run the Custom Script Extension on first boot

Reimaging discards any local data that is not stored on external disks or shared storage. Make sure application data is persisted outside the OS disk.

As a last resort, you can delete all existing instances and scale back up so that all new instances start with a clean state and the agent is installed.

bash

# Delete existing instances
az vmss delete-instances \
  --resource-group <your-resource-group> \
  --name <your-vmss-name> \
  --instance-ids "*"

# Scale back up (new instances will have the agent)
az vmss scale \
  --resource-group <your-resource-group> \
  --name <your-vmss-name> \
  --new-capacity <desired-number-of-vms>

Only use this approach when you can safely recreate all instances from the base image.

Note : Use an in-place update when you want the change applied across the scale set with minimal disruption. Once the rollout succeeds, the Custom Script Extension runs on the instances and installs the Middleware Agent.

4 Verify the agent installation#

After the rollout, connect to a VM instance and verify that the Middleware Agent is running:

Sign in to the Middleware platform.
Go to the Infra tab.
Confirm that your Azure VMSS instances appear in the list of virtual machines or hosts.

New VMs may take a few minutes to show up after the agent starts.

You can also verify the service on a VM directly:

bash

sudo systemctl status mw-agent

The service should show an active (running) state. If the service is not running or fails to start, check the agent logs under /var/log for more details.

Verify data collection#

After the Agent is running, go to Infrastructure Monitoring → Hosts and verify that the VMSS instances are reporting data.

After data collection starts, you can:

View host metrics in Infrastructure Monitoring.
View collected host logs in Log Monitoring.
Create alerts for collected metrics.
Create custom dashboards to visualize and monitor metrics from your VMSS instances.

Troubleshooting#

Check extension status in Azure

If the extension fails, first check the status at the VMSS level:

bash

az vmss extension list \
  --resource-group <your-resource-group> \
  --vmss-name <your-vmss-name>

Look for the CustomScript extension and review:

Provisioning state
Any error messages returned by the platform

For per instance status, you can also use az vmss get-instance-view to inspect a specific VM.

View Custom Script Extension logs on a VM

If the extension reports failure, SSH into one of the affected instances and check the handler logs:

bash

cat /var/log/azure/custom-script/handler.log

This log usually contains:

The command that was executed
Script output and errors
Exit code of the script

Common issues:

Missing or invalid MW_API_KEY or MW_TARGET
Network connectivity problems when curl tries to download the install script
The package manager is locked for longer than 300 seconds, which triggers the timeout in the script

Remove and re-add the Custom Script Extension

If the extension configuration is corrupted or you want to start from a clean state, remove it and add it again.

bash

az vmss extension delete \
  --resource-group <your-resource-group> \
  --vmss-name <your-vmss-name> \
  --name CustomScript

Then repeat Step 2 to re-add the extension

After re-adding, repeat the rollout steps for existing instances.

Check VMSS capacity and scale back up

If you deleted instances and no new VMs are created, confirm the scale set capacity and adjust it.

bash

# Check current capacity
az vmss show \
  --resource-group <your-resource-group> \
  --name <your-vmss-name> \
  --query '{capacity:sku.capacity, instanceCount:instanceCount}'

# Scale up if capacity is 0
az vmss scale \
  --resource-group <your-resource-group> \
  --name <your-vmss-name> \
  --new-capacity 3

Once new instances are created, they will use the VMSS model that already contains the Custom Script Extension and will install the Middleware agent automatically.

Need assistance or want to learn more about Middleware? Get in touch with us via our Contact Us or join our Slack channel.