The Express Storage Architecture (ESA) introduced in VMware vSAN 8 revolutionizes the implementation of compression. With ESA, VMware has redesigned and optimized the way compression is implemented, delivering enhanced storage efficiency and performance. In this blog post, we will explore the key aspects of ESA's compression implementation and how it improves the overall storage capabilities of vSAN 8.
New strategies are in place to boost the compression feature's potential data reduction to as high as an 8:1 compression ratio for every 4KB block written, which is a 4x improvement over the original storage architecture. The new compression functionality in vSAN is designed in the upper layers of vSAN as it receives incoming writes. It helps to reduce not just the amount of CPU effort required to compress the data, but also the reduces the amount of network traffic, because all replica traffic is always compressed. This very efficient design allows it to be enabled by default, but it can be turned off per VM using storage rules for workloads that already use application-level compression.
Why the shift?
vSAN OSA -
Compression was originally implemented in vSAN with the release of VSAN 6.2. This feature first came out in a world where hosts and resource costs looked radically different. Hosts with six core processors that lacked many modern acceleration features were widespread, and the cost of CPU overhead and potential latency or metadata amplification to a first write were heavily evaluated while building a system that compressed data solely on cache de-stage. Data was adaptively compressed based on its compressibility. This occurred at the expense of compression savings, but it did help minimize CPU overhead at a time when CPU resource budgets were comparable to today's. At the time vSAN data services were added at the “bottom of the stack” at the disk group layer and this design made sense.
vSAN ESA -
"Performance without compromise" is a core theme of vSAN ESA. Two important innovations have enabled a new compression method that is not only more efficient in terms of capacity (up to a 4x improvement over the old storage architecture), but also consumes less CPU. The vSAN file system along with compression “up the stack” to execute immediately on the host where the virtual machine is running.
Writes - The compression process only needs to be run once vs. once for each copy of the data. This reduces amplification of CPU and data written to the drives. This data being compressed before it hits the network also reduces the network traffic required.
Reads - Data is fetched in a compressed state, so this reduces the amount of data transmitted across the network.
Granular management without complexity -
With the vSAN original storage architecture compression was available as a single/simple datastore wide option. This made sense as it was deeply embedded in the lower levels of the IO path, and configuring the file system is an “all or nothing” option. It could be enabled or disabled after the fact but this forced a relatively heavy IO operation of re-writing all data to the cluster in a rolling fashion.
vSAN 8 Express Storage Architecture gives more flexibility and control. The new compression option is enabled by default, but can be optionally disabled for workloads though the use of the vCenter Storage Policy Based Management (SPBM) Framework. By adding a policy that uses “No Compression” you can disable compression to a specific disk or virtual machine. If disabled (or re-enabled) by policy, it will not change the compression state of existing data retroactively. Compression will only be effective on subsequent writes. This removes the operational performance impact considerations for enabling or disabling the policy. A few applications may perform their own compression (VMware Tanzu Greenplum/PostgreSQL, video, etc.). In these cases, forcing a disable of the compression will save CPU cycles. But this can be done easily and prescriptively using storage policies.
The Express Storage Architecture in VMware vSAN 8 provides an opportunity to improve data services without sacrificing performance or capacity costs. vSAN 8 optimizes storage capacity utilization while minimizing any impact on system performance. With these enhancements, organizations can achieve higher storage efficiency, cost savings, and seamless integration within their vSAN environment.
Thank you for reading!
*** Explore | Share | Grow ***