An HA cluster (high availability) with Proxmox VE ensures your virtual machines survive a hardware or node failure: if a server goes down, Proxmox automatically restarts the affected VMs on another, healthy node. This guide shows how such a cluster is built, what its requirements are and how to set it up step by step.
One important note up front: Proxmox HA is restart-based, not seamless. It minimizes downtime to the duration of a reboot, but does not replace cluster-wide "zero downtime". If you are looking for the basics, see What is Proxmox?.
For the distinction between HA and site recovery, see Proxmox disaster recovery with backups, replication and a runbook.
Cluster or HA: what is the difference?
A cluster joins several Proxmox nodes into one management domain: a central web interface, shared configuration through the cluster file system pmxcfs, migration of VMs and containers between nodes, and cluster-wide services such as the firewall and HA (Proxmox cluster documentation). HA is one of these services: the HA manager restarts VMs and containers that are configured as HA resources on another node after a node fails.
It follows that a cluster can run without HA, while HA requires a cluster. The HA documentation lists at least three cluster nodes for reliable quorum, shared storage for VMs and containers, and hardware redundancy as requirements (Proxmox HA documentation). The seven steps below build both.
Requirements
A production HA cluster needs a solid foundation:
- At least three nodes. Quorum (the cluster's ability to make decisions) requires the majority of votes. With three nodes, the cluster survives the loss of one node and stays operational. Two nodes only make sense with a QDevice as a third vote (more on this in step 3).
- A reliable, fast network. Cluster communication runs over Corosync and is sensitive to latency (Proxmox cluster documentation, network requirements). Latencies below 5 ms between nodes and a dedicated network (its own NIC) for Corosync only - separate from VM and storage traffic - are recommended. The nodes must reach each other over UDP ports 5405 to 5412. How to implement this on rented servers is covered in the guides for Hetzner, OVH and IONOS.
- Synchronized clocks and identical version. NTP on all nodes, identical Proxmox VE version, reachable SSH port 22 between nodes.
- HA-capable storage. The VM disks must be available on multiple nodes - via distributed shared storage (Ceph) or via ZFS replication.
- Adequately sized hardware. After a node fails, the HA manager distributes its guests across the remaining nodes; the documentation requires designing the cluster for that case. The guide on Proxmox hardware requirements and sizing covers sizing.
Example setup for this guide: three nodes pve1 to pve3 running Proxmox VE 9.2, a separate Corosync network and Ceph as storage.
| Network | Purpose | Example addresses |
|---|---|---|
| Management and VM | Web interface, VM traffic, SSH | 10.0.0.0/24 (pve1 = 10.0.0.1) |
| Corosync | Cluster communication, its own physical NIC | 10.10.10.0/24 (pve1 = 10.10.10.1) |
| Ceph | Storage replication, 10 GbE or faster | 10.20.20.0/24 |
Step 1: Create the cluster
Hostname and IP address of the nodes have to be final before you create the cluster: according to the documentation they cannot be changed afterwards, and neither can the cluster name. On the first node, create the cluster, here with the Corosync network as link 0 (without a separate cluster network, omit --link0):
pvecm create my-cluster --link0 10.10.10.1
You can check the status at any time with:
pvecm status
Step 2: Add further nodes
On each additional node, join the existing cluster. The first address is that of an existing node, and --link0 sets the node's own address on the Corosync network (Proxmox cluster documentation):
pvecm add 10.0.0.1 --link0 10.10.10.2
On the third node the link 0 address is 10.10.10.3 accordingly. All nodes then appear in the web interface under "Datacenter" and share their configuration through the cluster file system pmxcfs. Use pvecm status to check the join: with three nodes, the "Votequorum information" section shows three total votes, and the flags include "Quorate".
Step 3: Understand and secure quorum
In a network partition, only the nodes holding the majority of votes may keep working; the losing side goes read-only. Therefore:
- Three or more nodes are the clean solution.
- For a two-node cluster, add a QDevice: a lightweight service (
corosync-qnetd) on a third, independent system (e.g. a small VM on other hardware or a Raspberry Pi) that provides the deciding third vote. The complete setup with prerequisites and removal is described in the guide on setting up a two-node cluster with a QDevice.
apt install corosync-qdevice
pvecm qdevice setup <QDEVICE-IP>
Step 4: Storage for HA - Ceph or ZFS replication
The VM disks must survive the node failure. Two established approaches:
| Ceph | ZFS replication | |
|---|---|---|
| Principle | Distributed shared storage, synchronous | Asynchronous replication on an interval |
| Minimum nodes | 3+ | from 2 |
| Network | 10 GbE+ recommended | less critical |
| Data freshness after failure | current (RPO ~0) | state of the last replication (RPO = interval) |
| Complexity | higher | low |
| Best for | production with high data freshness | lean setups with tolerated data loss |
Rule of thumb: from three nodes with a fast network, Ceph is suitable as synchronous shared storage (Proxmox Ceph documentation). For small, cost-conscious setups, ZFS replication is a pragmatic near-HA solution - with the awareness that on failover the changes since the last replication are missing. The storage choice is covered in more depth in Proxmox storage: ZFS, Ceph or LVM; planning and operating Ceph on Proxmox is described on the page Proxmox Ceph.
Step 5: Enable HA
High availability is managed by two services: the Cluster Resource Manager (pve-ha-crm, one per cluster) and the Local Resource Manager (pve-ha-lrm, one per node) (Proxmox HA documentation). In practice:
- In the web interface under Datacenter → HA, add the desired VMs/containers as HA resources. On the command line this corresponds to
ha-manager add vm:100. - Optionally define HA rules: node affinity rules set which nodes a VM should preferably or exclusively run on (including priorities), and resource affinity rules keep VMs together or apart. The former HA groups have been deprecated since Proxmox VE 9.0 and are migrated to node affinity rules automatically (Proxmox HA documentation).
- Set the desired target state (e.g.
started).
From now on, the HA stack monitors the resources and automatically restarts them on a node failure. Since Proxmox VE 9.2, the Cluster Resource Scheduler in dynamic mode can also rebalance HA-managed guests automatically based on current load (as of October 2026). The ha-rebalance-on-start option selects the best-suited node when an HA resource starts and migrates the resource there if necessary; in the load-based modes (static-load and dynamic-load) the documentation still marks this feature as a technology preview. VMs are moved via live migration; LXC containers cannot be live-migrated and use restart migration instead, which stops, moves and restarts them with a short downtime (Proxmox container docs).
Proxmox DRS and load balancing explains CRS modes, automatic redistribution and their limits.
Step 6: Fencing and watchdog
To prevent a seemingly dead node from still running a VM while it starts elsewhere (split-brain), Proxmox uses watchdog-based self-fencing (Proxmox HA documentation, fencing), which needs no additional external hardware. If a node loses quorum, its watchdog expires and the node reboots itself before the HA resources start elsewhere.
By default the softdog kernel module is used, a software watchdog: if no hardware watchdog is available or configured, Proxmox falls back to it. The documentation rates it as reliable but not independent of the server hardware, and therefore less reliable than a hardware watchdog. Hardware watchdog modules are blocked by default for security reasons. To enable one, enter the module in /etc/default/pve-ha-manager, for example WATCHDOG_MODULE=iTCO_wdt (Proxmox HA documentation, hardware watchdog); which module fits depends on the server's mainboard.
Step 7: Test failover
An HA cluster is only as good as its tested failure case. Simulate a node failure (hard power-off or disconnect the node) and watch ha-manager status on a remaining node: according to the documentation, the state of the HA resource changes to fence (the HA manager waits until the failed node is fenced), then to recovery (search for a new node) and finally to started on another node (HA resource states). Also check with pvecm status that the cluster stays quorate, and note how long the VM was unreachable. Also document how to fence a node manually.
Maintenance in an HA cluster: maintenance mode and reboots
The HA stack also takes part in updates and hardware maintenance. The HA documentation distinguishes two approaches (Node Maintenance):
- Shutdown policy for shutdowns and reboots. You set it under Datacenter → Options → HA Settings (or in
datacenter.cfgviaha: shutdown_policy=<value>) to define what happens to a node's HA guests on shutdown or reboot. Withmigrate, the HA manager moves all HA guests to other nodes and delays the shutdown until they are running there. That requires the guests to be movable; with a local binding, such as hardware passthrough, they are not.conditionalis currently the default (Shutdown Policy): on a reboot the HA stack freezes the guests and starts them again on the same node after the restart, on a shutdown it stops them and other nodes take over. - Manual maintenance mode for work without a reboot, or work that should not end after a single reboot. The command marks the node as unavailable for HA, and all HA-managed guests move to other nodes:
ha-manager crm-command node-maintenance enable NODENAME
After the maintenance, ha-manager crm-command node-maintenance disable NODENAME turns the mode off; the guests that were on the node when the mode was enabled are moved back. The maintenance mode persists after a reboot of the node until you disable it. You can tell that the node has handed over its guests when no active HA resource remains on it.
For work on the network or the Corosync stack that may briefly cost quorum, the documentation also describes disabling HA (disarm-ha); while HA is disarmed, it does not protect any services.
The cluster update guide with HA and reboot describes the full procedure for regular per-node package updates.
Common mistakes
- Two-node cluster without a QDevice. On every failure the cluster loses quorum and goes read-only - HA then does not take effect reliably.
- Corosync over the storage or VM network. Latency spikes lead to quorum loss and unnecessary fencing actions. Corosync belongs on its own, stable network (ideally redundant).
- A node with existing VMs is supposed to join. Joining overwrites the configuration in
/etc/pve, and a joining node must not hold any guests. Before joining, back up each guest withvzdumpand restore it under a new ID afterwards (Proxmox cluster documentation). This also applies if the cluster only emerges later from existing single nodes. - Confusing HA with "zero downtime". HA means automatic restart, not seamless operation.
- No fencing planned. Without a watchdog or fencing, you risk data corruption from split-brain.
- Neglecting backups. HA replicates errors and ransomware too. A separate backup - for example with encrypted Proxmox backups on a Hetzner Storage Box or an offsite Proxmox Backup Server - remains mandatory.
Have the cluster set up or rent one
An HA cluster is manageable to set up but demanding to run: quorum design, the Corosync network, storage strategy, patch management and tested restores all have to fit together cleanly. Depending on your starting point there are four routes:
- Have the cluster built: Proxmox setup from a single node to a 3-node cluster.
- Rent a cluster: managed Proxmox hosting with servers and operations from a single source, also as a cluster.
- Have an existing cluster operated: Managed Proxmox for maintenance, monitoring and backup responsibility.
- Have Ceph looked after: Proxmox Ceph for planning and operating the distributed storage.
Managing several clusters centrally is a topic of its own: Proxmox Datacenter Manager for multiple clusters. Individual clusters retain their own quorum; PDM does not replace cross-cluster HA. If you are planning a single installation first, the way in is setting up Proxmox on Hetzner.
