
Several solutions exist, with limits. Adding a parity strip for triple parity reduces the odds of data loss. Global declustering, on its own, can also reduce the odds a little bit, with faster rebuild times. However, stripes are spread over every drive in the cluster, so two random failures are more likely to hit the same stripe, meaning a larger fault domain.
If both approaches are used, which is the minimum bar for him, you get down to getting struck by “lightning on your birthday” odds. VAST already publishes quadruple parity, which is beyond this bar. Using VAST’s own “special magic,” undisclosed but beyond the minimum bar, drops the odds even further, specifically for the petabyte SSDs, which is the real target of the talk.
Then we get to the metadata, the data about data, which in this case is the index of where every piece of every file dwells. Every write necessarily updates it synchronously, so VAST keeps it on separate, fast SSDs as three full copies for replication, not parity. This means it can survive two failures, which is less than the data’s four parity strips, and the index becomes the weakest link. With 122 TB drives, he puts the data drives’ odds at 1 in 78 trillion over 10 years, against 1 in 58 million for metadata. To Lockwood, this illustrates that the surrounding infrastructure is the real risk, given the failure gap between the index and regular data.
Rebuilding in the index area starts about 12 seconds after a failure, on average, versus about two minutes for data drives, Lockwood says. VAST’s procedure more generally, for when a drive drops out, is to power-cycle it and hope, so there is always some delay before a rebuild is forced. This slight delay barely matters for data drives with four parity strips but is more impactful for the three-copy metadata index drives. For example, waiting an hour for data drives “wouldn’t make a big difference” while just 20 minutes or more on a metadata drive has “extremely high” risk.
Earlier this year, in May, Micron began shipping the 245 TB 6600 ION , which pulls up to 30 W at maximum load. Kioxia has the LC9 , used in a Dell 2U server with 40 drives for a total of 9.8 PB. SanDisk also has its 256 TB UltraQLC, which was due in the first half of this year, and Solidigm is targeting about 245 TB later this year .
Meanwhile, multiple roadmaps reach petabyte-class drives, including SanDisk’s UltraQLC , a target by SK hynix for QLC SSDs, and Samsung at some time in the future. This strengthens VAST’s case and, while at least the power math adds up, data still needs to be compared openly and fairly to make a firm conclusion about the best pathway forward for what is clearly an unavoidable problem.
Shane Downing Social Links Navigation Contributing Writer Shane Downing is a Contributing Writer for Tom’s Hardware, covering consumer storage, PC hardware, and AI.
DS426 ZFS wasn't even mentioned? I realize it's not a 1:1 replacement for RAID, but it does fix a lot of issues that RAID has (bit rot, don't have to worry about hardware RAID controller failures, etc.). I personally think RAID solves a problem from an era that we've largely grown out of. In some deployments, it's still makes a lot of sense and where ZFS isn't an option. I do think we need a mental paradigm shift to a more software-defined storage drive and data redundancy future. On large scales, this would result in a lot less "wasted" drives while simultaneously reducing the chances of data loss. Hardware and traditional software RAID would still have their use cases but not be the de facto standard. Reply
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/pc-components/ssds/SPONSORED_LINK_URL
- https://www.tomshardware.com/pc-components/ssds/vast-data-says-petabyte-ssds-will-outgrow-raid-in-todays-parallel-file-systems-simulation-of-a-900-pb-system-puts-data-loss-odds-at-1-in-11-over-10-years#main
- https://www.tomshardware.com/my-account
- NVIDIA and Microsoft Expand Accelerated Computing From Azure to Windows PCs
- Lambda Boosts AI Factory Resiliency and Efficiency With NVIDIA Photonics
- How the World’s Most Advanced AI Infrastructure Supply Chain Invents the Future
- Three 'e-waste' Nvidia V100s hit 100 tokens per second on Qwen 3.6 35B in a $1,042 homelab rig
- Nvidia promises employee pay raises, equity before December to quell unrest in wake of Jensen Huang scandal
Informational only. No financial advice. Do your own research.