Abstract:
Linux kernel founder Linus Torvalds is developing a Linux kernel performance optimization solution with the help of artificial intelligence programming tool Claude Code.
This work targets a performance bottleneck in the Linux kernel memory management mechanism. The goal is to reduce competition caused by memory page reference counting when multiple threads read the same file at the same time, thereby improving file reading efficiency.
Torvalds not only used AI to assist in writing code, but also used this tool to write test programs, conduct code reviews, and draft relevant patch notes.
While waiting for the release of Linux 7.3-rc7, Torvalds recently refreshed his Linux kernel Git code repository and discovered that an unusual new branch had appeared in the repository. Unlike his usual practice of maintaining the main branch, this branch named "noref-filemap-read-with-llm" is dedicated to developing a series of kernel improvement patches. As can be seen from the LLM in the branch name, this work is related to large language model-assisted programming.
Torvalds later made it clear in the relevant patch notes that this work was completed under his guidance and with the help of Claude Code. AI not only participated in code implementation, but also assisted in writing test programs, conducting code reviews, and drafting the initial version of the patch series documentation. The relevant performance measurements were done on Torvalds' own computer.
This optimization mainly targets a specific problem in the Linux kernel file reading path. When multiple threads read small chunks of data from the same file at the same time, they may compete around the reference count of memory pages in the file page cache, causing threads to compete with each other for related data structures, thereby limiting read performance.
To understand this problem, you need to first understand how the Linux kernel handles file reading.
Linux will use the page cache (Page Cache) to save file data read from the storage device. When a program again accesses the contents of a file that has been cached in memory, the kernel can often provide the data directly from memory without having to revisit the disk or other storage device each time. Page caching is therefore an important mechanism for Linux to improve file access efficiency.
In order to manage these cached data, the kernel needs to track the life cycle of related memory objects. The reference counting mechanism in traditional implementations can help the kernel determine whether a memory object is still in use and when it can safely release related resources.
However, when a large number of threads frequently access the same file, even if the data they read is already in memory, the related reference counting operations may still incur additional synchronization overhead. In a multi-core processor system, if multiple threads continuously update the same shared count value, problems such as increased cache coherence traffic and processor cores waiting for each other may occur.
For single-threaded or low-concurrency scenarios, this overhead may not be noticeable. But in highly concurrent servers, databases, build systems, and other workloads that need to handle a large number of file read requests simultaneously, recurring synchronization operations may gradually become a performance bottleneck.
The series of patches Torvalds is developing are designed to reduce unnecessary reference counting operations. The core idea is to adjust the file read path so that the kernel can read data directly from the page cache under certain conditions without having to perform traditional folio reference management operations for each relevant access.
folio here is one of the basic abstractions used by the Linux kernel to manage a set of contiguous or associated memory pages. In recent years, the Linux kernel has gradually used the folio mechanism to replace some of the old single-page management methods in order to handle memory management tasks more efficiently. How to reduce the overhead caused by folio reference counting has therefore become an important task in kernel performance optimization.
From a technical perspective, this type of optimization does not simply remove reference counting. The kernel must still ensure that memory objects are not released prematurely while they are being accessed, and must not introduce data corruption or memory safety issues by reducing synchronization operations. Therefore, patches need to strike a balance between correctness, concurrent access safety, and performance.
This optimization resulted in quite significant performance changes under the target workload. The test is mainly aimed at the scenario where multiple threads read small pieces of data from the same file at the same time, which corresponds to the competition problem that the patch is trying to solve.
However, these results should be understood as performance under specific test conditions, not that all file operations on Linux will receive the same improvement. Actual benefits depend on your application's access patterns, number of threads, file size, storage device, memory configuration, and other system factors. Optimization effects may be limited for workloads that do not involve the same contention issues.
Currently, these patches are still stored in Torvalds’ personal Git branch, and it cannot be concluded that they have officially entered the Linux mainline kernel. Whether it will be merged in the future and what implementation method will be adopted in the end requires further observation.
According to the current development pace, this patch series may enter the further discussion or review stage during the next Linux 7.4 merge window, but whether it can be merged ultimately still depends on code review, test results and the judgment of the maintainer.
It is worth noting that Torvalds' use of Claude Code is not his first attempt to use artificial intelligence to handle Linux-related development work.
Previously, he has talked about using AI-assisted kernel debugging and tried AI programming tools such as Google Antigravity outside of the Linux kernel. For Torvalds, AI is not about replacing developers as independent decision-makers, but rather a development tool that can be used with clear goals and human supervision.
Since this year, artificial intelligence programming tools are gradually entering the open source software development process. More and more developers are leveraging large language models to analyze code, locate bugs, write tests, and generate initial patches. The Linux kernel community is also seeing an increasing number of AI-assisted code and bug reports.
This trend brings both opportunities and challenges. On the one hand, AI can help experienced developers explore complex code paths faster, come up with optimization ideas, and reduce the time required to write repetitive code. On the other hand, a large number of automatically generated patches and bug reports also increase the review burden on maintainers, and some of the submissions may have problems of low quality, duplicate submissions, or lack of actual value.
Torvalds has publicly stated before that he believes AI is a useful tool, but the kernel community needs to deal with the work pressure caused by the rapid increase in AI-generated content. He does not advocate rejecting code just because it uses AI, but pays more attention to the quality, correctness and actual effect of the final code.
The kernel optimization project that Claude Code participated in this time reflects a more specific application method of AI-assisted development: developers who are familiar with the internal mechanisms of the kernel identify problems, propose optimization directions and supervise the implementation, and then verify the results through testing and code review.
Unlike relying solely on AI to automatically generate the entire software project, Linux kernel development involves complex concurrency mechanisms, memory safety requirements and hardware behavior. Any seemingly simple modification can affect a large number of applications and different hardware platforms. Therefore, even if AI can help generate code, developers still need to understand how it works and take responsibility for the final implementation.
If this set of patches finally passes review and enters the Linux mainline, it will become another case of AI-assisted development participating in the optimization of underlying system software. Its value lies not only in the performance improvements that may be obtained in specific scenarios, but also in demonstrating how large language models can assist experienced engineers in analyzing and improving complex systems.
As AI programming tools continue to develop, more similar work may appear in the Linux kernel and other large open source projects in the future. However, what determines whether these achievements can actually be implemented will still be strict technical verification, community review, and long-term requirements for system stability.
Comments