Abstract:
Meta’s recently launched AI assistant Muse was exposed to a serious zero-day security vulnerability. Muse for macOS has extremely broad system permissions. Once exploited, an attacker can obtain the token used to verify the identity of the Muse account through local applications or even terminal commands, and further control the entire AI assistant account.
Security researchers said that this means that attackers can perform a large number of high-risk operations with the permissions that Muse itself has obtained, including writing malicious files, taking photos, and reading user data.
Muse is a new AI agent recently launched by Meta. It can complete operations such as making appointments, filling out forms, handling customer service matters, and shopping on behalf of users. It can also generate pictures, create documents, and connect to users' commonly used applications and online services. Currently, Muse provides a macOS version, and a Windows version has not yet been released. In order for Muse to complete these tasks, users need to grant it access to WhatsApp, email, calendar, and social media accounts.
Unlike ordinary chatbots, Muse is an AI agent that can actually perform operations on behalf of the user. It can even dynamically create the tools you need while executing a task. Therefore, Muse must obtain far wider system permissions than traditional AI chat applications.
On macOS, Muse needs to obtain a series of permissions protected by the operating system, including writing files to disk, accessing the microphone and camera, obtaining location, and accessing the calendar. The reason why Apple designed these system permission restrictions is to prevent ordinary applications or programs executed in the terminal from calling these sensitive resources at will.
However, security researchers discovered that Muse's design actually bypasses some of the security isolation mechanisms originally provided by macOS.
The vulnerability was discovered by macOS security expert Patrick Wardle. He found that any locally installed application or executing code, no matter how limited macOS permissions it has, can modify a large number of undisclosed internal settings of Muse.
The vast majority of these settings themselves do not pose an obvious security risk, such as changing user interface options such as dark mode. But one setting is crucial because it allows the process to modify the network endpoints used by Muse for speech transcription.
Under normal circumstances, Muse will send the speech transcription request to the server operated by Meta. However, an attacker can take advantage of the vulnerability and change this address to a server that he controls. Once Muse starts sending requests to the malicious server, the token used to authenticate the user's Muse account may also be obtained by the attacker.
Once this token is obtained, the attacker no longer controls just one voice request, but can gain continuous control over the entire Muse account. Wardle said that attackers can directly use the high privileges that Muse has obtained to complete various operations without having to specially write a complex set of macOS malware.
Wardle has produced multiple proof-of-concept attacks, including using Muse to write malicious files to disk and call the camera to take photos. In some tests, even a very alert user might not see obvious security warnings.
This means that Muse has a special security problem: the attacker does not necessarily need to first obtain complete control of Muse itself, but only needs to find an entry point that allows malicious code to run on the Mac, and may further exploit the system privileges that Muse has obtained.
One type of attack in particular is worth noting, the so-called ClickFix attack. ClickFix has become a very effective means of social engineering attacks in recent years. Its basic method is to trick users into performing seemingly normal operations or commands, but in fact they run malicious code provided by the attacker on the device.
Wardle said that with a simple modification of this attack method, it is possible to further control the Muse account. This also makes the conventional wisdom that "all security measures are meaningless if your Mac has been compromised" not entirely applicable to Muse.
The reason is that the consequences of attacks on ordinary applications and attacks on AI agents are different. Muse itself has obtained a large number of permissions to access user data and perform actual operations. Therefore, as long as an attacker can use Muse to complete permission escalation, an original local attack with very limited permissions can be transformed into large-scale control of the AI agent.
An attacker can also use network proxies to launch attacks. One way is to have an attacker-controlled server between the Muse user and the Meta server. When a user enters a voice command into Muse, an attacker can insert a malicious prompt into the request to induce Muse to perform the operation the attacker wants to complete, such as requiring Muse to package all the user's WhatsApp messages and send them to the attacker.
What's more serious is that once Muse's authentication token is also sent to a malicious server, the attacker may gain continuous control over the Muse account instead of just completing an attack.
Wardle believes that multiple design decisions at Muse combined to create the vulnerability. One of the key issues is Meta's choice to let Muse complete speech transcription in the cloud.
macOS itself has long provided mechanisms for completing dictation and transcription locally on the device. If Meta chose to keep sensitive voice data inside the device, the attacker's attack by modifying the address of the cloud transcription server would not be feasible.
Another problem is that Muse allows any local application to control a large number of undisclosed settings. Wardle believes that Meta may have originally just wanted to allow applications that collaborate with Muse to adjust user interface related parameters, and this design itself has a certain rationality. But allowing any application to alter the server endpoints that handle sensitive voice data is an entirely different security risk.
Wardle believes these design decisions raise a larger question about how much security considerations Meta put into designing and testing Muse. He said that for AI applications with such broad system permissions, security requirements should be much higher than ordinary software.
Meta has previously published two consecutive articles detailing the measures Muse has taken for privacy and security during the design process. Meta founder and CEO Mark Zuckerberg has also emphasized that Muse has been designed in accordance with privacy and security requirements from the beginning.
However, the exposure of the zero-day vulnerability is in clear contrast with the security concept that Meta has previously emphasized. Especially in the context of recent security incidents that have occurred in other AI models, the issue of AI agents gaining more and more practical operating permissions is attracting the attention of security researchers.
Previously, Anthropic and Google models had security incidents involving external third-party networks during internal testing. Although the tests were not intended to attack these networks, the ability of AI systems to act autonomously has sparked ongoing discussions in the security field.
At the same time, Amazon began blocking Muse from shopping on its website about 12 hours before the vulnerability was made public. When users try to ask Muse to shop on Amazon, they will see a prompt from Amazon saying that Muse is an unauthorized AI agent and violates Amazon's terms of use.
Amazon said third-party apps that enable purchases from other businesses on behalf of customers should operate openly and transparently and respect the service provider's decision to allow them to participate in transactions. Amazon believes that this is similar to the relationship between takeout platforms and restaurants, delivery platforms and stores, and online travel agents and airlines. AI agents that can perform transactions on behalf of consumers also need to abide by this principle.
Amazon also asked Meta to remove its platform from Muse’s shopping experience.
The restrictive measures taken by Amazon this time also occur in the context of the competition between Meta and Amazon around AI agent shopping. In the future, AI agents will be able to directly browse websites, select products and complete payments for users. Therefore, how to identify traditional websites and whether to allow AI agents to access them is becoming a new business and technical issue.
Meta has yet to respond to specific questions raised by the media about this zero-day vulnerability, so it is unclear whether the company has developed a patch, whether it has begun pushing fix updates to affected users, and whether this vulnerability was actually exploited before being discovered by researchers.
Wardle said he plans to further introduce this vulnerability and discuss other security threats that AI assistants may pose at the Objective by the Sea security conference in November this year. He also believes that the security standards of AI agents must be significantly higher than that of ordinary applications, because in order to complete tasks authorized by users, such software often needs to access accounts, communications, files, cameras, microphones and other sensitive resources at the same time.
The problems exposed by Muse this time also show that there are obvious differences in the security models of AI agents and traditional applications. Even if vulnerabilities occur in traditional software, attackers usually still need to gradually obtain system permissions; AI agents themselves are designed to perform operations on behalf of users. Therefore, once there are flaws in its authentication mechanism or permission boundaries, attackers may directly use the AI agent's originally legally obtained permissions to complete high-risk operations.
The specific impact scope of this vulnerability and the progress of Meta's repair still need to be further confirmed. But for users who need AI agents to connect to email, instant messaging, calendaring, social media, and payment and shopping services, the Muse incident once again highlights a core issue: the more permissions an AI assistant has, the greater the importance of its own security mechanism.
Comments