A perfect illustration of an optimization for memory, after an optimization for speed
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works .
"Backend routing," or the mapping of a URL to one of many cached servers, requires a lot of tables held in memory. An incoming URL is hashed (like you do with files with CRC-32), and that resulting number is taken and sent out to a server. The first instinct is to do this sequentially, one URL per server, but you hit the first problem: the same URLs aren't going to the same servers. So what you do is you make a hash corresponding to the server, say with its IP address and name, and match both hashes together by numerical proximity.
That sounds simple, and it works fine, as the URL hashes' distribution is going to be effectively random, and the requests will be redirected to the same server in an egalitarian way. But then you run face-first against the second problem: say you have four servers serving 25% of requests each, and #2 disappears because someone tripped on the power cable. You'll be forced to redirect the requests it handled onto whichever are closest in the hash map, and end up overloading server #3, the next one over, while #1 and #4 remain under little load.
The solution to that is to add more hashes per server and mix them up randomly. Now your four servers' many hashes have a random distribution, and if one dies, incoming requests should be equally distributed among the remaining ones. This takes a lot of memory, and it takes far more once you set up layers with weight rules (so that bigger servers handle more requests), and the fact that, due to content restrictions and architectural regions, not all servers can serve all requests. All told, Cloudflare was running with as many as 100,000 server hashes per machine, ballooning RAM requirements.
After judicious application of some algebra and basic statistics, Cloudflare engineers concluded that using those 100,000 hashes was far and away beyond the point of diminishing returns. The team calculated that a mere 10% of that amount was good enough for nearly the same results, as the error rate barely drops for each order of magnitude beyond 10,000. After some judicious wrangling of Rust's data structures to save 2 bytes per entry in the hash-server map. That sounds like nothing, but it quickly adds up with billions of records.
Cloudflare frees up 100TB of RAM by shrinking 1.1.1.1's DNS cache entries
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/software/SPONSORED_LINK_URL
- https://www.tomshardware.com/software/cloudflare-saves-100-tb-of-ram-again-this-time-by-slashing-server-hashes-by-90-percent-cutting-100-000-entries-down-to-10-000-eliminates-massive-cache-bloat#main
- https://www.tomshardware.com/membership
- Cloudflare saves 100 TB of RAM again, this time by slashing server hashes by 90% — cutting 100,000 entries down to 10,000 eliminates massive cache bloat
- d-Matrix Adopts NVIDIA NVLink Fusion for Rack-Scale XPU Deployment
- This week on Tom's Hardware Premium: September 19, 2026 — Steam Frame interview, killer AI models, and the DRAM crisis deepens
- Give your PC the deep clean it deserves — the Wolfbox MF60 Air Duster with up to 110,000 RPM drops to $33.99
- Noctua fans prevent the CAIM1 ‘Anti-AI’ 4K camera from throttling — unusual camera’s processor gets toasty as it records while performing cryptographic calculat
Informational only. No financial advice. Do your own research.