Review storage architecture for your cluster: Proxmox consulting for the ZFS vs Ceph decision or Ceph setup.
The right storage choice in Proxmox VE depends mainly on whether you run a single server or a cluster. In short: for a single node, ZFS is the best all-round choice (snapshots, compression, checksums, software RAID), and LVM-thin is the lean alternative for setups with little RAM. For a cluster with high availability across three or more nodes, Ceph is the standard because it distributes and shares storage across all nodes. Directory and NFS round things out for ISOs, templates and backups. This article explains the options, their strengths and weaknesses, and gives recommendations per scenario.
Which filesystem for Proxmox?
Proxmox VE supports several storage types (Proxmox storage documentation) that fall into two broad classes: local storage (available on a single node only) and shared storage (shared across multiple nodes). The decision follows simple logic:
- One server, no high availability: ZFS or LVM-thin.
- Cluster with real HA and shared storage: Ceph (distributed) or external shared storage (NFS, iSCSI).
- Holding ISOs, container templates and backups: Directory or NFS.
The key point: there is no single best filesystem. The question is always which behaviour you need and which hardware you have.
What is ZFS?
ZFS is a combined filesystem and volume manager, and the most feature-rich local storage in Proxmox. It has been the recommended choice for new single-node installations for years. The most important properties:
- Snapshots and clones: Consistent point-in-time copies of VMs and containers in seconds, without doubling storage use. Ideal before updates and for fast clones.
- Replication: ZFS can replicate datasets to another node on an interval via
zfs send/receive. This is the basis for ZFS-based near-HA in Proxmox. - Compression: Enabled by default with
lz4. It often saves space and can even improve performance through fewer physical writes. - Data integrity: ZFS keeps a checksum for every block. That lets it detect silent data corruption (bit-rot) and repair it automatically in redundant pools (mirror, RAIDZ) through self-healing.
- Built-in software RAID: Mirror, RAIDZ1/2/3 without a separate RAID controller.
As of October 2026, Proxmox specifies at least 8 GB RAM as a starting point for ZFS. Its ARC rule of thumb is at least 2 GiB base plus 1 GiB per TiB of storage. The installer defaults to an ARC limit of 10 percent of host RAM, capped at 16 GiB (source). Under memory pressure, OpenZFS can shrink the ARC to its configured minimum (source). Two practical rules: ZFS belongs directly on the disks (HBA / IT mode), not behind a hardware RAID controller, and ECC RAM is recommended for maximum integrity.
For deployment and acceptance, the Proxmox Ceph setup and health-check guide covers nodes, networks, pools and health checks.
What is Ceph?
Plan and build Ceph and check hardware and sizing.
Ceph is a distributed object store integrated into Proxmox VE as hyper-converged shared storage. Instead of keeping data locally on one server, Ceph distributes it, replicated, across all nodes in the cluster. If a node fails, the data remains available on the others. That is exactly what makes Ceph the standard foundation for high availability and HA clusters with Proxmox.
The requirements are considerably higher than for local storage, though:
- At least three nodes. Only from three nodes onward can Ceph distribute data redundantly in a meaningful way. Three is the minimum of fault tolerance; four to five nodes are more robust.
- A fast, dedicated network. According to the Proxmox Ceph documentation, at least 10 Gbit/s exclusively for Ceph is recommended, and at least 25 Gbit/s with NVMe drives. Latency and bandwidth directly determine storage performance.
- Multiple drives per node. Each disk becomes an OSD (object storage daemon); more OSDs mean more performance and redundancy.
Ceph is powerful but not a set-and-forget system. A Ceph deployment that is too small or poorly networked is a common cause of performance and stability problems.
What are LVM and LVM-thin?
LVM (Logical Volume Manager) is the classic, proven block storage layer of Linux. Two variants matter in Proxmox:
- LVM (thick): Reserves a volume's full capacity immediately. Robust and simple, but without thin provisioning.
- LVM-thin: Allocates only the blocks actually written (thin provisioning) and supports snapshots and clones. This is the usual choice for VM and container disks and the default on LVM-based installations.
LVM-thin is the lean alternative to ZFS: minimal RAM needs, very robust, predictable performance. What it lacks are checksums against bit-rot and built-in self-healing. LVM-thin is local and cannot be shared across nodes. Classic LVM can instead use shared block storage such as iSCSI. New since Proxmox VE 9: classic and shared LVM (for example over iSCSI or Fibre Channel) can create snapshots through qcow2-based volume chains. As of Proxmox VE 9.2 (June 2026) this feature is still a technology preview, however, and not released for critical production data.
Directory, NFS and iSCSI
Directory is file-based storage on an ordinary filesystem. It is always available (the default local storage) and suits ISO images, container templates and backups. VM disks are stored here as qcow2 files, which also support snapshots but can be slower on network storage.
NFS is file-based shared storage: an external NAS or file server that presents the same path to all nodes. NFS enables live migration without a local copy and is a simple shared-storage option when Ceph is not wanted. However, it is a central single point of failure unless the NFS server itself is built redundantly.
iSCSI exposes external block storage over an IP network and can provide shared storage. Proxmox commonly uses LVM over the supplied LUN to divide it into VM volumes; include redundancy and multipathing in the design (storage documentation).
Comparison table: ZFS vs Ceph vs LVM vs Directory/NFS
| ZFS | Ceph | LVM-thin | Directory / NFS | |
|---|---|---|---|---|
| Type | Filesystem + volume manager | Distributed object store | Block storage | File storage |
| Shared (cluster) | No (local) | Yes | No (local) | NFS: yes, Directory: no |
| Snapshots | Yes (native) | Yes | Yes | Yes (qcow2) |
| Checksums / self-healing | Yes | Yes | No | No |
| Compression | Yes | Yes | No | Only via qcow2 |
| Software RAID | Yes (mirror, RAIDZ) | Yes (distributed) | No | No |
| RAM needs | High (ARC) | Medium to high | Low | Low |
| Minimum nodes | 1 | 3+ | 1 | 1 (NFS external) |
| Network | uncritical | 10 GbE+ (25 GbE for NVMe) | uncritical | depends on NAS |
| Fits | single-node all-round | HA cluster | lean single-node | ISOs, templates, backups |
Recommendations: which storage to use when
- Single server, you want comfort and safety: ZFS. Snapshots, compression and checksums pay off daily and before updates. Budget the RAM and use an HBA instead of hardware RAID.
- Single server, little RAM or maximum simplicity: LVM-thin. Robust, frugal, predictable - without the ZFS comfort.
- Cluster of three or more nodes with real HA: Ceph, provided the network keeps up (10 GbE+). Plan for four to five nodes rather than the bare minimum.
- Two-node cluster with QDevice or small budget with near-HA: ZFS replication. Simpler than Ceph, with the understanding that failover loses the changes since the last replication.
- ISOs, templates, backups: Directory locally or NFS for shared access.
One final, important point: no storage layout replaces a backup. ZFS replication and Ceph mirror mistakes and ransomware too. A separate, ideally offsite backup, for example with encrypted Proxmox backups on a Hetzner Storage Box, remains mandatory.
Further reading: What is Proxmox Backup Server?.
Plan and operate your storage
Review your storage architecture or plan, build and operate Ceph. Book an initial consultation.
For the procedure and limitations see the ZFS replication guide using pvesr.
Sources
As of October 2026. Basis are the ARC limit, installer default, iSCSI and local-lvm of the vendor documentation. The ARC default applies to new installations; check the actual configuration on older installations.
