Cloudflare Cut 100TB From a DNS Cache. Here's the Playbook

Cloudflare shaved 100TB off 1.1.1.1's DNS cache. The trick: attack per-entry overhead. Measure your own cache's multiplier in 60 seconds.

Dafallah Ibrahim

— 2 min read
Share

Cloudflare just published how they saved 100 terabytes of memory on 1.1.1.1's DNS cache. Not by adding machines or evicting harder — by optimizing what the cache itself costs. A hundred terabytes is a fleet-scale number, but the same tax applies to your in-process LRU, your Redis, your edge cache: most caches burn several times more memory than the data they actually hold.

The takeaway generalizes to everything you run. Before you provision more memory, measure what one cache entry really costs you.

Why this matters

Caches feel like "data in, data out," but between your data and the allocator sits a stack of hidden costs. At a million entries, nobody notices. At a hundred million, the overhead is the workload. Cloudflare's number shows the ceiling: at scale, memory optimization isn't polish, it's capacity planning.

There's a latency dividend too. Compact entries mean better CPU cache locality. Shrinking a hash map often makes lookups faster before you touch the lookup code.

How it works

1. Measure bytes per entry. RSS delta divided by entries inserted. If you can't quote that number for your hottest cache, you're not doing capacity planning — you're doing vibes.

2. Find where the bytes go. The usual suspects: node-based containers (std::unordered_map, chained hash tables) burn two to three pointers per entry; malloc headers and alignment round every small allocation up (glibc's smallest chunk is 32 bytes); hash tables keep spare slots for load factor; GC languages add object headers — a Python str costs 49 bytes before its first character.

3. Attack the overhead. Swap node containers for open-addressing maps (abseil flat_hash_map, Rust's hashbrown) that store entries inline. Replace pointers with 32-bit indices into a slab. Deduplicate repeated keys and values — DNS names share suffixes like ".example.com", so store that once. Trade exact LRU for clock or S2-LRU eviction and drop the two-pointer linked list per entry.

Where this helps

  • Edge caches keyed by full URL, where a million long keys quietly dominate node memory.
  • In-process session and rate-limit maps in Go or Rust services that grow with traffic.
  • DNS and IP-prefix lookups, where radix tries prefix-compress keys almost for free.
  • Metrics label sets — interning repeated label combinations routinely shrinks cardinality-heavy caches by 5x.

Watch out

Compact layouts cost flexibility. Slabs and inline storage make deletes, resizing, and iteration awkward. Truncating keys to hashes invites collisions — a DNS cache must return exact answers, so you still store enough key to verify every hit. GC languages fight you too: you can't control object layout, so the fix usually means moving the hot cache into a native structure or a serialized blob. And measure first — this work only pays past a scale threshold.

Try it yourself

Linux only. Watch a plain dict turn ~34 MB of data into ~200 MB of RSS:

python3 - <<'EOF'
import resource

def rss_mb():
    # Linux: ru_maxrss is reported in KB
    return resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024

before = rss_mb()
cache = {}
for i in range(1_000_000):
    cache[f"edge-{i:07d}.example.com"] = "203.0.113.%d" % (i % 254)
after = rss_mb()

data = 1_000_000 * (24 +

Read more