Abstract:
On November 4, Addy Osmani, an engineer on the Claude Code team of Anthropic and a former Google technology executive, recently published a long article on the social platform He calls this phenomenon
"cognitive surrender"
(Cognitive Surrender).
Osmani has worked at Google for more than 14 years and has been responsible for Chrome developer experience and Google Cloud AI related businesses. He is currently working on Claude Code at Anthropic.
Osmani said that he is a staunch supporter of AI agent workflow and often runs multiple agents at the same time in his daily work. But he discovered a change that deserves vigilance:
As AI models continue to improve, there are fewer and fewer obvious errors in the output results, making it harder for humans to stay vigilant.
In the past, people usually needed to go through the process of information collection, analysis, judgment, etc. to get the answer, but now, this process may be compressed to a few seconds. AI-generated content is often well-structured, fluently expressed, and correct in most cases, making it increasingly easy for users to skip independent thinking and verification.
He divided the state of human use of AI into three levels: The first is
"cognitive offloading"
(Cognitive Offloading), which means the task is handed over to AI for completion, but it is still responsible for the results; the second is"postponing judgment"
(Deferral), that is, the AI output is no longer regarded as a hypothesis that needs to be verified, but directly used as the basis for the next step of work; the last is"Cognitive Surrender"
,That is, the output of AI gradually becomes the user's own judgment, and the user no longer even thinks it is necessary to check.
Among them, the concept of "cognitive surrender" was first proposed by Wharton School researchers Steven Shaw and Gideon Nave.
Osmani believes that
this transformation often does not happen suddenly, but gradually forms in the process of continuous use of AI
.
Users may read the first AI report carefully, only take a quick glance at the second one, and even just check the outline of the third one. Eventually, when asked about a specific conclusion, users may find themselves unable to explain the logic.He linked this phenomenon to "cognitive debt." In the field of software development, AI can generate a large amount of code, and all tests may pass, but team members' understanding of the system architecture and implementation logic is constantly decreasing. It wasn't until one day when the system needed to be modified that the developers discovered that no one really understood how certain features worked.
However, Osmani doesn't believe developers need to remember all the details of a huge code base. He pointed out that the Claude Code team will also use AI to re-establish understanding of specific code modules when needed. What's really important is not knowing everything all the time, but knowing what you don't know and having the ability to re-understand it.
As for how to avoid "cognitive capitulation," Osmani suggested,
Before asking complex questions to AI, humans should form their own preliminary assumptions instead of relying entirely on AI to provide answers.
Even if the initial guess is not accurate, it can help users retain a basis for independent judgment when receiving AI results.He also suggested that
users should actively look for evidence that can overturn the AI's conclusion, rather than just asking the model how confident it is in its answer.
For controversial issues, different models can present arguments from opposite positions, and then humans can make comprehensive judgments.In the field of software development, Osmani believes that the bottleneck in the AI era is shifting from writing code to verifying whether the code is correct.
Currently, AI agents can independently write code, run tests, read error messages and modify them repeatedly until the test passes. But the problem is that the tests themselves may also be written by AI. If the test logic and code have the same errors, even if all tests pass, it cannot prove that the software does not have problems.
Therefore, he emphasized that developers should not only pay attention to whether the test passes, but also need to clarify what the test proves and what risks are not covered. For the results delivered by AI, the acceptance criteria should be verifiable evidence such as actual test results, operation logs, screenshots, etc., rather than a seemingly reasonable summary generated by AI.
Osmani also proposed that developers should require the Agent to clearly list the decisions it made autonomously during the execution of the task, the assumptions it adopted, and the conditions it ignored, because these contents usually do not appear in the final work summary.
As AI is able to perform more and more tasks simultaneously, another problem begins to arise. Osmani calls it a
"orchestration tax"
(Orchestration Tax), that is, the gap between the speed at which AI generates results and the speed at which humans can carefully review and be willing to take responsibility for them.When the work results generated by AI continue to accumulate and humans no longer have enough time to review them, continuing to increase the number of Agents will not solve the problem. He believes that in this case, the team should improve the automatic verification mechanism or appropriately reduce the task generation speed.
Osmani also warned that
AI's high degree of automation may cause humans to gradually give up practicing their original skills.
He recommends that developers try to write some code without using AI every week, revisit the skills they have stopped practicing every month, and judge which abilities are still worth retaining.He also cited Anthropic's research on the use of AI by internal engineers and researchers, pointing out that there is a "supervision paradox": Supervising Claude requires programming abilities, but excessive use of AI may degrade these abilities.
Osmani emphasized that these views do not mean that the use of AI should be reduced. There is no problem in tasks where the cost of error is low, completely handed over to AI, but when it comes to investments, career choices, product safety, or major decisions that affect other people, humans should still retain key judgment rights.
In his view, future AI models may be better at finding errors than humans, but this does not eliminate human responsibility.
Deciding what products to develop, why they are worth developing, who they will impact, and what price they are willing to pay for it still requires human choices.
He concluded at the end of the article: "
You can hand over the work and most of the review tasks to AI, but you cannot hand over the responsibility as well.
”
Comments