How to reduce write amplification on NAND chips in a cloud storage system
Solid-state disk (SSD) technology using integrated circuit assemblies is considered to be a readily available, drop-in replacement for discrete hard disks and is used to increase throughput in large cloud storage networks. But unfortunately, these devices introduce the problem of write amplification (WA) into a cloud storage system.
Reducing this occurrence is of paramount concern to most system administrators who use NAND chips in these situations because it reduces the lifespan of these chips and harms the throughput rate of a cloud storage and retrieval paradigm.
Calculating write amplification values
All forms of NAND memory chips used in SSD construction have to be erased before they can be written to again. This means that moving, rewriting and storing data to pages on SSDs will cause used portions of the chips to go through the process more than once for each file system operation. Consider a drive that has several logical blocks filled with data and several other continuous blocks that are empty. An operator using a Web-interface document manager adds data to a spreadsheet, and then saves it from his/her client machine. This necessitates the logical block that holds it to be erased and then rewritten.
If the file now takes up more space than a single block provides, other used blocks will have to be erased and rewritten until the file system achieves stability. Added to this wear and tear is the leveling and garbage collection necessary on NAND devices. Write amplification can be seriously problematic. Fortunately, there is a way to calculate the amount of WA that will be necessary to store data from cloud clients: total wear amplification = (amount of data written to NAND chips / data the host writes)
Overprovisioning and logical capacity
The difference between the actual physical capacity of an SSD and the logical capacity that an operating system sees is referred to as “over-provisioning” (OP). Additional space provided by an OP design helps the controller chip handle wear-leveling and garbage collection tasks, and provide additional blocks after some go bad
The most obvious and superfluous source of this comes from system software that uses a binary Gigabyte (1,024 MB = 1 GB), while hardware manufacturers utilise the metric Gigabyte (1,000 MB = 1 GB). This figure usually isn’t relevant when considering the problem of write amplification, but a second OP source comes from hardware vendors that purposefully add unused NAND chips somewhere in an SSD. If these values are known, OP values can be calculated easily: OP = (true capacity – visible capacity) / total user capacity
Cooperative data management policies
Several different new cooperative data management policies can be installed across a cloud storage network to cut down on WA cycles. Eliminating redundant cloud storage system caches is the first step. Currently, file systems often keep additional caches with the hopes of speeding the system up, but this only increases speed when working with mechanical drives. Ultimately, this increases granularity by orders of magnitude on SSD devices.
These policies also create victim blocks, which aren’t written to and are instead skipped, because they’re marked as “in use” in the cache. Utilising file systems designed to work with NAND chips changes these policies, which also help to reuse these victim blocks. Most importantly, the cache system should be flushed out, ensuring that extra data isn’t written to the drive.
Leveraging page metadata to reduce file system overhead
Traditional file systems that rely on an allocation table, journal or master record protect all file metadata to ensure durability (even in the event of system failures). This requires numerous writes to these tables, and while it provides an extra layer of safety, new file systems designed to work with solid-state memory are starting to replace these models.
Metadata can be leveraged to guarantee that indices can be restored if anything unusual happens, as opposed to ensuring that the metadata is always durable. By using a consistent approach instead of looking at durability to solve the problem, far less metadata has to be written to the drive to begin with. This dramatically reduces the influence of WA cycles.
Write amplification must be reduced
System administrators need to reduce write amplification, as it adds overhead and reduces the life of chips. As new file systems designed with NAND technology mature, compliance from the industry is likely to rise as the advantages become clearer.
- » Waste not, want not: How enterprises can avoid an idle cloud estate
- » Cloud complexity and ‘terrifying’ IoT means organisations’ asset visibility is worsening – report
- » What is cyber insurance truly worth? Analysing the risks and responses
- » How cloud providers are changing the outlook for IoT data and analytics management
- » Blog: How cloud companies are reacting to Covid-19 and services offered: AWS, Alibaba, and more