Installing the Middleware agent on Azure Virtual Machine Scale Set (VMSS)

Use this guide to install the Middleware host agent on every virtual machine in an Azure Virtual Machine Scale Set (VMSS) with the Azure Custom Script Extension.

Once you attach the extension to the scale set model, Azure will:

  • Run your installation script on all new instances created by the scale set
  • Optionally run the same script on existing instances, so the rollout is consistent

This approach is ideal when:

  • You already have a Linux VM scale set running your workloads
  • You want the Middleware agent to be present on every VM without manual SSH
  • You want future scale out events to automatically include the agent

How it works

At a high level:

  1. You create a shell script that:

    • Detects the operating system
    • Selects the correct package manager and install script
    • Waits for existing package manager operations to finish
    • Installs the Middleware agent using your API key and target URL

  2. You attach this script to your VMSS using the Azure Custom Script Extension.

  3. Azure runs this script:

    • During extension deployment on existing instances (when you update them)
    • Automatically on every new instance that the scale set creates

If the extension fails or the VM image changes, you can reimage or recreate instances so they pick up the latest extension configuration and agent version.

Prerequisites

Before you start, make sure you have:

1 Middleware Account

  • An active Middleware account and project
  • Your Middleware API key (MW_API_KEY) and target URL (MW_TARGET)

2 Azure Resources

  • An existing Linux VM scale set in Azure
  • Azure CLI installed and logged in with sufficient permissions to:
    • Read and update the VM scale set
    • Add, update and delete VMSS extensions

3 Supported Distributions

These instructions target Linux distributions that use apt, yum, dnf or zypper.

Installation

1 Prepare the installation script

Create a file named install-middleware-agent.sh on the machine where you run Azure CLI.

1#!/bin/bash
2
3# Detect OS and package manager
4
5echo "Detecting operating system..."
6
7if [ -f /etc/os-release ]; then
8    . /etc/os-release
9    OS=$ID
10    echo "Detected OS: $OS"
11else
12    echo "ERROR: Cannot detect OS"
13    exit 1
14fi
15
16# Determine package manager and installation script
17
18if command -v apt-get >/dev/null 2>&1; then
19    # Debian/Ubuntu
20    PKG_MGR="apt"
21    INSTALL_SCRIPT="https://install.middleware.io/scripts/deb-install.sh"
22    LOCK_FILES="/var/lib/dpkg/lock-frontend /var/lib/apt/lists/lock /var/lib/dpkg/lock"
23elif command -v yum >/dev/null 2>&1; then
24    # RHEL/CentOS/Amazon Linux
25    PKG_MGR="yum"
26    INSTALL_SCRIPT="https://install.middleware.io/scripts/rpm-install.sh"
27    LOCK_FILES="/var/run/yum.pid"
28elif command -v dnf >/dev/null 2>&1; then
29    # Fedora/RHEL 8+
30    PKG_MGR="dnf"
31    INSTALL_SCRIPT="https://install.middleware.io/scripts/rpm-install.sh"
32    LOCK_FILES="/var/cache/dnf/metadata_lock.pid"
33elif command -v zypper >/dev/null 2>&1; then
34    # SUSE/openSUSE
35    PKG_MGR="zypper"
36    INSTALL_SCRIPT="https://install.middleware.io/scripts/rpm-install.sh"
37    LOCK_FILES="/var/run/zypp.pid"
38else
39    echo "ERROR: Unsupported package manager"
40    exit 1
41fi
42
43echo "Using package manager: $PKG_MGR"
44echo "Installation script: $INSTALL_SCRIPT"
45
46# Wait for package manager locks to be released
47
48echo "Checking for package manager locks..."
49
50timeout=300
51elapsed=0
52
53while true; do
54    locked=false
55    for lock_file in $LOCK_FILES; do
56        if [ -f "$lock_file" ] && fuser "$lock_file" >/dev/null 2>&1; then
57            locked=true
58            break
59        fi
60    done
61
62    if [ "$locked" = false ]; then
63        break
64    fi
65
66    if [ $elapsed -ge $timeout ]; then
67        echo "ERROR: Timeout waiting for package manager locks (waited ${timeout}s)"
68        exit 1
69    fi
70
71    echo "$PKG_MGR is busy, waiting... (${elapsed}s elapsed)"
72    sleep 10
73    elapsed=$((elapsed + 10))
74done
75
76echo "Package manager available. Installing Middleware agent..."
77
78# Install Middleware agent
79
80MW_API_KEY="<MW_API_KEY>" \
81MW_TARGET="https://<MW_UID>.middleware.io:443" \
82bash -c "$(curl -L $INSTALL_SCRIPT)"
83
84if [ $? -eq 0 ]; then
85    echo "Middleware agent installed successfully"
86    exit 0
87else
88    echo "ERROR: Middleware agent installation failed"
89    exit 1
90fi

What this script does:

  • Detects the OS by reading /etc/os-release
  • Chooses the correct package manager (apt, yum, dnf or zypper) and the matching Middleware install script
  • Waits for package manager locks to clear, which avoids conflicts with other system updates
  • Runs the Middleware installer with MW_API_KEY and MW_TARGET set as environment variables
  • Returns a non-zero exit code if installation fails, so the Azure extension can report failure

Save the file and make it executable:

1chmod +x install-middleware-agent.sh

2 Attach the Custom Script Extension to the VM scale set

Run this command from the directory where install-middleware-agent.sh is located:

1az vmss extension set \
2  --resource-group <your-resource-group> \
3  --vmss-name <your-vmss-name> \
4  --name CustomScript \
5  --publisher Microsoft.Azure.Extensions \
6  --version 2.1 \
7  --protected-settings "{\"script\": \"$(base64 -w 0 install-middleware-agent.sh)\"}"

What this command does:

  • Adds the Custom Script extension to the VMSS model
  • Embeds the base64 encoded install-middleware-agent.sh into the extension configuration
  • Ensures future instances created by the scale set automatically run this script during provisioning

Existing instances will not run the script until you update, reimage or recreate them in the next step.

If you run this command from a non-Linux environment, use the equivalent base64 command that produces a single line output with no line breaks.

3 Roll out the agent to existing VM instances

After the extension is attached to the scale set model, you need to make sure existing instances also pick up the change.

You can choose one of the following rollout strategies:

Update instances in place (recommended)
Rolling upgrade for production
Reimage instances
Delete and recreate instances

Try this first. It refreshes the configuration of all instances and triggers the extension without recreating VMs.

1az vmss update-instances \
2  --resource-group <your-resource-group> \
3  --name <your-vmss-name> \
4  --instance-ids "*"

Use this when:

  • You are fine with instances being updated in place
  • You want the change to apply across the scale set with minimal disruption

If this succeeds, the Custom Script Extension runs, the agent installs and you can move to verification.

Use a rolling upgrade when you want to apply the new extension configuration gradually across the scale set to reduce impact on production workloads.

1az vmss rolling-upgrade start \
2  --resource-group <your-resource-group> \
3  --name <your-vmss-name>

During a rolling upgrade, Azure:

  • Updates a subset of instances at a time
  • Waits for the instances to become healthy
  • Proceeds to the next batch

Even with a rolling upgrade, VMs are restarted. Any state that is not externalised will be lost on upgraded instances.

If you see errors like PropertyChangeNotAllowed or configuration drift, reimage the instances so they boot from the base image and apply the latest extension settings.

1az vmss reimage \
2  --resource-group <your-resource-group> \
3  --name <your-vmss-name> \
4  --instance-ids "*"

This action:

  • Recreates the OS disk for each selected instance
  • Keeps the scale set configuration and networking
  • Causes the instance to run the Custom Script Extension on first boot

Reimaging discards any local data that is not stored on external disks or shared storage. Make sure application data is persisted outside the OS disk.

As a last resort, you can delete all existing instances and scale back up so that all new instances start with a clean state and the agent is installed.

1# Delete existing instances
2az vmss delete-instances \
3  --resource-group <your-resource-group> \
4  --name <your-vmss-name> \
5  --instance-ids "*"
6
7# Scale back up (new instances will have the agent)
8az vmss scale \
9  --resource-group <your-resource-group> \
10  --name <your-vmss-name> \
11  --new-capacity <desired-number-of-vms>

Only use this approach when you can safely recreate all instances from the base image.

4 Verify the agent installation

After the rollout, verify that the Middleware agent is running.

In the Middleware UI
On a VM instance
  1. Sign in to the Middleware platform.
  2. Go to the Infra tab.
  3. Confirm that your Azure VMSS instances appear in the list of virtual machines or hosts.

New VMs may take a few minutes to show up after the agent starts.

You can also verify the service on a VM directly:

1sudo systemctl status mw-agent

You should see the service in an active (running) state. If the service is not running or fails to start, check the logs under /var/log for more details.

Troubleshooting

Check extension status in Azure

If the extension fails, first check the status at the VMSS level:

1az vmss extension list \
2  --resource-group <your-resource-group> \
3  --vmss-name <your-vmss-name>

Look for the CustomScript extension and review:

  • Provisioning state
  • Any error messages returned by the platform

For per instance status, you can also use az vmss get-instance-view to inspect a specific VM.

View Custom Script Extension logs on a VM

If the extension reports failure, SSH into one of the affected instances and check the handler logs:

1cat /var/log/azure/custom-script/handler.log

This log usually contains:

  • The command that was executed
  • Script output and errors
  • Exit code of the script

Common issues:

  • Missing or invalid MW_API_KEY or MW_TARGET
  • Network connectivity problems when curl tries to download the install script
  • The package manager is locked for longer than 300 seconds, which triggers the timeout in the script

Remove and re-add the Custom Script Extension

If the extension configuration is corrupted or you want to start from a clean state, remove it and add it again.

1az vmss extension delete \
2  --resource-group <your-resource-group> \
3  --vmss-name <your-vmss-name> \
4  --name CustomScript

Then repeat Step 2 to re-add the extension

After re-adding, repeat the rollout steps for existing instances.

Check VMSS capacity and scale back up

If you deleted instances and no new VMs are created, confirm the scale set capacity and adjust it.

1# Check current capacity
2az vmss show \
3  --resource-group <your-resource-group> \
4  --name <your-vmss-name> \
5  --query '{capacity:sku.capacity, instanceCount:instanceCount}'
6
7# Scale up if capacity is 0
8az vmss scale \
9  --resource-group <your-resource-group> \
10  --name <your-vmss-name> \
11  --new-capacity 3

Once new instances are created, they will use the VMSS model that already contains the Custom Script Extension and will install the Middleware agent automatically.

Need assistance or want to learn more about Middleware? Contact our support team at [email protected] or join our Slack channel.