PostgreSQL Integration

The PostgreSQL integration enables you to ingest performance and health metrics from your PostgreSQL instance(s) into Middleware, allowing you to track blocks read versus hits, table/index sizes, transactions (including commits/rollbacks), per-statement activity, and more.

Prerequisites#

Setup#

1 Enable pg_stat_statements#

pg_stat_statements exposes per-statement stats that the integration relies on.

  • Edit postgresql.conf. (path example below; adjust for your distro/version)
    # /etc/postgresql/14/main/postgresql.conf
      shared_preload_libraries = 'pg_stat_statements'
      pg_stat_statements.track = all
      pg_stat_statements.max   = 10000
      track_io_timing          = on
    This ensures the extension is preloaded and IO timing is captured for accurate statement metrics. Save the file.
  • (Alternative via SQL): As a superuser, you can also set the preload parameter from SQL (still requires a restart to take effect):
    ALTER SYSTEM SET shared_preload_libraries = 'pg_stat_statements';
  • Restart PostgreSQL for the preload to take effect:
    sudo systemctl restart postgresql
  • Verify the extension and data flow (as superuser):
    CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
    SELECT calls, query FROM pg_stat_statements LIMIT 1;

You should see a row like:

calls | query
------+-------------------------------
    8 | SELECT * FROM t WHERE field = ?

(Your values will differ. This view reflects your executed statements.)

2 (optional): Create a least-privileged user#

If you don’t want to use the superuser for monitoring, create a read-only user with just what’s needed:

  • Create the user
    CREATE USER lpu WITH PASSWORD 'pass';
    Verify:
    \du
    You should see lpu in the roles listing.
  • Grant minimal access
    -- allow connections to the database you'll monitor
    GRANT CONNECT ON DATABASE <your_db_name> TO lpu;
    
    -- permit reading statement stats
    GRANT SELECT ON pg_stat_statements TO lpu;
    
    -- permit reading global stats where needed
    GRANT pg_read_all_stats TO lpu;

Test with the lpu account:

SELECT calls, query FROM pg_stat_statements LIMIT 1;

(The output will reflect queries observed under your workload.)

3 Open the Integration in Middleware#

In Middleware, go to Installations → All Integrations → PostgreSQL.

Postgres Overview

4 Fill the Integration Form#

In the PostgreSQL integration form:

  • Select the host (from the dropdown) where the database is reachable.
  • Enter the required connection details:
    • Database name
    • Endpoint
    • Username
    • Password
  • (Optional) Query Collection:
    • Query Sample Collection captures sampled query activity (query text, state, wait events, and client/application details) for troubleshooting.
    • Top Query Collection captures aggregated high-impact query metrics (calls, rows, block I/O, and planning/execution timings) from pg_stat_statements.
  • (Optional) Schema Collection captures database schema and table metadata for schema visibility in Middleware.
  • TLS Settings:
    • If Allow Insecure is false and Skip Certificate Verification is false, provide:
      • Client certificate file path
      • Client key file path
      • CA certificate file path

5 Save and Enable the Integration#

  • Click Save.
    Postgres Host

Once saved, the integration starts ingesting metrics.

Visualise your Data#

Default PostgreSQL Dashboard#

After setup, Middleware adds a PostgreSQL dashboard to the Dashboard Builder. Use it as a starting point to explore key DB metrics without building widgets from scratch.

Create Custom Widgets#

When adding a widget, choose data source postgresql to browse all exposed metrics and craft charts for your SLOs or run-book checks.

Alerts#

You can alert on any PostgreSQL metric. Create a rule using Database as the detection method and PostgreSQL as the database type; the Metrics dropdown then lists all available metrics for this integration. Configure conditions, thresholds and recipients as usual.

Metrics Collected#

The following metrics are emitted when enabled in the PostgreSQL receiver (names and descriptions align with the integration’s metric definitions).

Core Engine & Storage#

Metric NameDescription
postgresql.backendsThe number of backends.
postgresql.blocks_readThe number of blocks read (tagged by block read source).
postgresql.buffer_hitThe number of disk block hits in the buffer cache, thereby avoiding database reads, tagged with database name.
postgresql.db_sizeThe database disk usage.
postgresql.database.countNumber of user databases.

Tables, Indexes & Maintenance#

Metric NameDescription
postgresql.table.countNumber of user tables in a database.
postgresql.table.sizeDisk space used by a table.
postgresql.table.vacuum.countNumber of times a table has manually been vacuumed.
postgresql.table_bloatEstimated table bloat ratio (actual pages / expected pages); 1.0 indicates no bloat.
postgresql.analyzedNumber of times a table has been manually analyzed.
postgresql.autoanalyzedNumber of times a table has been automatically analyzed.
postgresql.autovacuumedNumber of times a table has been automatically vacuumed.
postgresql.index.scansThe number of index scans on a table.
postgresql.index.sizeThe size of the index on disk.
postgresql.index.blocks_readThe number of disk blocks read by this index (tagged by source).
postgresql.index.rows_readThe number of index entries returned by scans on this index.
postgresql.index_bloatEstimated index bloat ratio (actual pages / expected pages); 1.0 indicates no bloat.

TOAST#

Metric NameDescription
postgresql.toast.blocks_hitNumber of TOAST block hits.
postgresql.toast.index.blocks_readNumber of TOAST index block reads.
postgresql.toast.sizeSize of TOAST table.

Transactions & Row Activity#

Metric NameDescription
postgresql.commitsThe number of commits.
postgresql.rollbacksThe number of rollbacks.
postgresql.rowsThe number of rows in the database (tagged by row state).
postgresql.rows_deletedRows deleted by queries in this database, tagged with relation name.
postgresql.rows_fetchedRows fetched by queries in this database, tagged with relation name.
postgresql.rows_insertedRows inserted by queries in the database, tagged with relation name.
postgresql.rows_updatedRows updated by queries in the database, tagged with relation name.
postgresql.operationsThe number of database row operations (tagged by operation: ins, upd, del, hot_upd).

Checkpointing & Background Writer#

Metric NameDescription
postgresql.bgwriter.checkpoint.countThe number of checkpoints performed (tagged by checkpoint type).
postgresql.bgwriter.durationTotal time spent writing and syncing files to disk by checkpoints (tagged by duration type).
postgresql.bgwriter.maxwrittenNumber of times the background writer stopped a cleaning scan because it had written too many buffers.
postgresql.bgwriter.buffers.writesNumber of buffers written (tagged by buffer source).
postgresql.bgwriter.buffers.allocatedNumber of buffers allocated.

Connections & Limits#

Metric NameDescription
postgresql.connection.countThe number of active connections to this database; when DBM is enabled, tagged with state, application name, database, and user.
postgresql.connection.maxConfigured maximum number of client connections allowed.

Statement-level (via pg_stat_statements)#

Metric NameDescription
postgresql.query.countNumber of times the statement was executed (tagged with query_text, query_id).
postgresql.query.total_exec_timeTotal wait time of the normalised timed events (nanoseconds; tagged with query_text, query_id).

Replication & WAL#

Metric NameDescription
postgresql.replication.data_delayThe amount of data delayed in replication (tagged with replication client).
postgresql.wal.ageAge of the oldest WAL file (requires WAL with at least one replica).
postgresql.wal.countNumber of WAL files.
postgresql.wal.lagTime between flushing recent WAL locally and receiving notification that the standby completed an operation (tagged by operation and replication client; requires WAL with at least one replica).
postgresql.wal.sizeTotal size of WAL files.

Live Row Estimates#

Metric NameDescription
postgresql.live_rowsThe approximate number of live rows, tagged with relation name.

Active Transaction Duration#

Metric NameDescription
postgresql.transactions.duration.maxMax duration of active transactions.
postgresql.transactions.duration.sumSum of duration of active transactions.

AWS RDS PostgreSQL#

If you use AWS RDS for PostgreSQL, monitoring is host-agent-based: run the Middleware Host Agent on an EC2 instance that can reach your RDS endpoint and complete the same PostgreSQL integration steps in Middleware. For RDS-specific setup (parameter groups, connection details in AWS, and networking), see the AWS RDS PostgreSQL integration.

Azure Database for PostgreSQL#

If you use Azure Database for PostgreSQL, monitoring is host-agent-based: run the Middleware Host Agent on an Azure VM that can reach your PostgreSQL server endpoint and complete the same PostgreSQL integration steps in Middleware. For Azure-specific setup (server parameters, firewall rules, and networking), see the Azure Database for PostgreSQL integration.

GCP Cloud SQL for PostgreSQL#

If you use GCP Cloud SQL for PostgreSQL, monitoring is host-agent-based: run the Middleware Host Agent on a GCE VM that can reach your Cloud SQL endpoint and complete the same PostgreSQL integration steps in Middleware. For Cloud SQL-specific setup (database flags, SSL client certificates, and networking), see the GCP Cloud SQL for PostgreSQL integration.

Troubleshooting#

“Integrations” menu not visible

If you don’t see Integrations in Middleware, your account probably lacks Installation permissions. Ask an admin to add Installation to your role in Settings.

Need assistance or want to learn more about Middleware? Contact our support team at [email protected] or join our Slack channel.