Abstract:
The Linux kernel is undergoing a major upgrade to the widely used LZ4 compression algorithm. Developers have recently submitted a set of large-scale patches, hoping to completely reconstruct the LZ4 implementation in the kernel, which will not only bring significant performance improvements, but also end the long-term separation and maintenance of the code from the upstream project.
At present, the LZ4 code in the Linux kernel is actually an independently maintained fork version, and its basic code remains roughly in the state from 2017 to 2018. Over the years since then, the LZ4 upstream project has continued to make a large number of bug fixes, performance optimizations, and code improvements, but these updates have not fully entered the Linux kernel.
This upgrade was led by Samsung engineer Michal Wilczynski. According to the submitted patch plan, Linux will no longer maintain a specially modified LZ4 fork version, but will instead directly introduce and synchronize the upstream LZ4 source code. In other words, the future Linux kernel will be able to more easily obtain the latest LZ4 compression and decompression improvements.
The developers said that this adjustment will not only reduce the maintenance workload, but also help avoid compatibility and synchronization problems caused by long-term forks. When the LZ4 project releases new optimizations or fixes in the future, the Linux kernel will be able to more easily absorb these updates directly.
Judging from the performance test results, the benefits brought by this upgrade are quite obvious.
In the test of the EROFS read-only file system using LZ4 compression, when using 4KB data blocks, the read throughput increased by about 11%; when using 64KB data blocks, the read throughput increased by up to 14%.
In addition to EROFS, other Linux kernel internal LZ4 tests also show further improvements. Some benchmark test results show that the performance improvement achieved by the new version of LZ4 can reach up to 21%.
LZ4 is one of the most common high-speed compression algorithms in the Linux ecosystem. It is widely used in file systems, memory compression, startup images, container environments, and various scenarios that require fast compression and decompression. Compared with algorithms that pursue higher compression rates, the biggest features of LZ4 are extremely low latency and extremely high throughput, so it has a large number of applications in performance-sensitive fields.
Industry insiders believe that although this update is not as eye-catching as new processor support or the addition of large-scale features, it is of great significance to the underlying performance of Linux. Since LZ4 is widely used by many subsystems, even a single-digit percentage optimization will bring considerable benefits over the long term.
At the same time, the Linux kernel has recently continued to promote optimization of compression technology. In addition to LZ4, the Zstd compression code is also receiving improvements and performance enhancements. As the size of data continues to grow and the demand for AI, cloud computing and container deployment expands, the importance of high-performance compression technology is becoming increasingly prominent.
If this set of patches is finally accepted by the mainline kernel, Linux will have a more modern, efficient, and easier-to-maintain LZ4 implementation in the future, and the kernel version that has been stuck in the status quo a few years ago will officially become history.
Comments