Abstract:
Cloudflare recently significantly reduced the cache memory used by its 1.1.1.1 public DNS resolver, freeing up approximately 100TB of memory across the entire network. This optimization does not involve replacing server hardware or removing memory modules, but is achieved by improving the software storage method of caching DNS data.
This work mainly revolves around the Big Pineapple. Big Pineapple is Cloudflare's caching platform that supports the 1.1.1.1 resolver, DNS firewall, and many other DNS services. Cloudflare system engineer Sebastiaan Neuteboom introduced in a technical blog five adjustments to the Rust code base that reduced the space occupied by a single typical DNS cache record from 953 bytes to 420 bytes.
The optimized system running speed has also been improved. Cloudflare said that cache insertion throughput increased from 625,000 items per second to 893,000 items per second, and query latency dropped from 828 nanoseconds to 670 nanoseconds. Over the course of the deployment from mid-May to early July, 99th percentile memory usage per instance dropped from 9.3GB to 5.3GB.
Cloudflare claims that Big Pineapple caches more than 250 billion DNS records at all times. At such a large scale, even a tiny amount of memory wasted per record adds up to a significant amount. The company estimates that for every unnecessary byte occupied in a cache entry, the entire server cluster consumes more than 250GB of additional memory.
One of the important changes involves the Vec and String types in Rust. These containers are designed to dynamically expand, so in addition to the actual data, additional information such as capacity is saved. However, the DNS record does not need to be expanded after it is entered into the cache. Cloudflare then replaced these structures with fixed-size boxed slices, removing metadata that was no longer useful. According to the company, this adjustment alone saved more than 15TB of memory.
Engineers also redesigned the way DNS responses are arranged in memory. Previously, Big Pineapple would save three sets of record lists for each response; the new design puts all records into the same buffer, and then uses a two-byte offset to identify the location of each record.
The cache now also no longer retains owner names when they are simply duplicates of the domain name being queried, but is instead rebuilt when the response is returned. Additionally, the system instead saves record data using raw DNS line format bytes with a length prefix, solving the wasted space issue in earlier designs. Previously, an IPv4 A record, which only took up 4 bytes, took up 144 bytes, as did the relatively rare NAPTR record, the largest DNS record type cached by Cloudflare. Switching to a variable-length raw data format eliminates the need for shorter records to reserve the same space as the longest records.
The 100TB of memory Cloudflare reclaimed across the entire server cluster is roughly equivalent to the memory capacity of 130 13th generation servers. Each of these servers is equipped with 768GB DDR5-6400 memory. Cloudflare considered a 1152GB memory configuration for this generation of servers, but ultimately gave up after evaluating the cost of higher-capacity memory.
With this memory space, Cloudflare plans to expand the DNS cache size. Keeping more records locally is expected to improve cache hit rates and reduce request traffic to authoritative DNS servers.
This is not the first time Cloudflare has carried out large-scale memory efficiency optimization. Last year, the company also rewrote the FL2 layer responsible for processing requests into Rust code. With server memory prices rising, unlocking the potential of existing infrastructure through software optimization may be more cost-effective than simply adding hardware.
Comments