Abstract:
Apple is developing a new application called Harbor that aims to turn Mac computers equipped with Apple Silicon chips into private AI inference servers. Unlike traditional local AI software, Harbor can not only run large language models on a single Mac, but can also connect multiple Macs through the Thunderbolt 5 interface, integrating the unified memory of different devices into a larger computing resource pool, thereby running AI models that a single computer cannot independently accommodate.
According to MacRumors report on November 10, this application is currently being tested by Claris, an Apple subsidiary, and has not yet been officially opened to the public. Claris's most famous product is the FileMaker database platform for enterprise users, so Harbor may be mainly targeted at enterprises, developers, and professional users who need to handle sensitive data in the future.
Harbor's core positioning is to enable users to deploy open weight artificial intelligence models on their Mac computers without relying on cloud servers. According to the currently disclosed product introduction, all AI reasoning processes can be completed on local hardware, and documents, questions and other data submitted by users do not need to be uploaded to external cloud services.
This is of particular importance to enterprise users. Many companies want to use large language models to analyze internal documents, write code, or process business information, but are unwilling to send customer data, financial information, and business secrets to third-party AI service providers. Harbor attempts to simplify the local deployment process by allowing enterprises to directly use their own Mac hardware to run AI models.
The most noteworthy feature of this application is that it supports the establishment of a computing cluster composed of multiple Macs through the Thunderbolt 5 interface.
Apple has added Remote Direct Memory Access (RDMA) support based on Thunderbolt 5 in macOS Tahoe 26.2. With this technology, Macs connected together can access memory resources provided by other devices with low communication overhead, creating conditions for distributed AI inference.
For example, a Mac equipped with 64GB of unified memory may not be able to independently load certain language models with large parameter sizes. However, if multiple Macs with sufficient memory are connected, Harbor can coordinate the use of the memory and computing power of these devices to run models that would otherwise exceed the capacity of a single device.
However, this kind of memory pooling does not mean that the memory of multiple Macs will become a storage space that is completely equivalent to the unified memory of the local machine at the hardware level. Cross-device access still requires transmitting data through the Thunderbolt interface, whose latency and bandwidth are different from the unified memory architecture inside the chip. Therefore, actual performance still depends on how the model is distributed to different devices and how the system coordinates computing tasks.
In addition to running larger models, multi-machine clusters can also increase concurrent processing capabilities. For enterprises that need to provide AI services to multiple employees at the same time, multiple Macs can share inference requests without having to rely entirely on a single high-configuration workstation.
Harbor also attempts to solve the long-standing complex configuration problem in local AI deployment.
Currently, although users can run open weight models on Mac through tools such as LM Studio and Ollama, if they want to connect multiple computers into a cluster, they usually need to install additional software, configure the network, manage model sharding, and solve communication problems between different devices.
According to the disclosed function description, Harbor can automatically identify Mac devices connected through Thunderbolt 5 and assist in establishing a cluster. Users only need to connect the device and start the application, and the system will automatically complete the necessary configuration. There is no need to use the command line or have professional server management experience.
In terms of model management, Harbor also provides a graphical operation interface, allowing users to directly browse, download and load open weight models. The application handles model file management and run configuration, allowing users to focus on actual work.
This design is equivalent to combining the ease of use of LM Studio with the multi-machine distributed reasoning capabilities. For users who are not familiar with complex AI infrastructure, if the final product can achieve these functions, the threshold for local deployment of large language models will be significantly lowered.
However, Harbor is not the first project to try to connect multiple Macs to run AI models. An open source tool called Exo has previously appeared, which can use multiple devices to jointly perform AI inference tasks. The more important difference of Harbor developed by Apple this time may be its integration with the underlying functions of macOS and its provision of a more simplified operating experience for ordinary enterprise users.
Currently Harbor is still in the early testing stage. Previously, the app briefly appeared in the App Store, but the related page was subsequently removed. Apple has yet to officially announce when the product will be released, or whether it will ultimately be available for free.
Harbor's early access application page requires a company name, further suggesting that Apple may launch the product for the enterprise market first, rather than as a general tool that all Mac users can immediately download.
From a hardware perspective, this application may also further expand the application scenarios of Mac computers with high memory capacity. Apple's unified memory architecture allows the CPU and GPU to share the same memory pool, so when running large language models, Mac Studio and Mac mini equipped with large amounts of unified memory have become the choice of some developers to deploy local AI.
If Harbor can allow multiple Macs to run together conveniently, enterprises can gradually add equipment according to needs without having to purchase a higher-configuration and more expensive independent workstation each time.
However, it should be noted that the main function currently disclosed by Harbor is AI inference, not the complete training of large models. Inference mainly involves loading a trained model and generating answers, while training requires a lot of additional calculations, gradient synchronization, and parameter update operations. There are obvious differences in the requirements for hardware and communication systems.
This technology does not mean that any two Macs can directly form a high-performance AI cluster. According to existing product introductions, the multi-machine memory pooling function relies on Thunderbolt 5, so only Macs equipped with compatible interfaces can use this connection method. For older devices that don't support Thunderbolt 5, there is currently no evidence that Harbor will provide the same clustering capabilities.
If Harbor is finally officially launched, it will mean that Apple begins to participate more directly in the local AI infrastructure software market. In the past, Apple mainly provided hardware and system foundations for third-party AI development tools through Apple Silicon chips and macOS; while Harbor may allow Apple to provide a complete set of solutions from model downloading and deployment to multi-machine operation.
This model has practical appeal for enterprises that want to run AI within their internal network and avoid sensitive data leaving local devices. However, Apple or Claris still need to announce more details about which models Harbor can ultimately support, how many Macs can form a cluster, what the actual inference performance is, and whether it will adopt a charging model.
Comments