The rapid development of AI technology has made many users inseparable from AI in their daily work. However, most of the current AI is based on the cloud, which means that the data submitted by users has potential leakage risks. At this time, the Mouzhi Foundation, which is concerned about privacy, stepped forward. The Mouzhi Foundation announced the launch of a new open source project called Llamafile. The goal of this open source project is to help people run large language models on their own hardware. It does not require a dedicated GPU, CPU or NPU (but it can run better with a GPU).
Llamafile works by compressing all the complexity of an artificial intelligence chatbot into a single file, which can then run on Windows, Mac, Linux, OpenBSD, FreeBSD, NetBSD, and requires almost no installation or configuration, that is, it can be used out of the box.
This kind of LLM is completely run locally and does not require cloud computing power provided by third-party commercial companies. The data is also saved locally and will not leave the user's device. If the user is willing, he or she can copy the robot model after daily conversations to other machines for continued use without re-optimizing the prompt words.
At present, Mozhi Foundation is not the only company that wants to build local AI. The Gemini variant AiCore recently provided by Google to Pixel 8 Pro is actually a local AI that uses the computing power of the local device to perform certain operations. However, there is currently relatively little public information about AiCore.
Mozhi Foundation believes that open source is one of the most powerful solutions to solve various privacy problems. It can solve the problems of privacy, access and control through open source, that is, who can hear our conversations? How will our data be processed? Who gets to decide whether the AI answers or not answers certain questions.
As for the Llamafile project, the Mozhi Foundation has noticed some promising projects. The first is the Llama.cpp project. This open source project allows ordinary consumer-grade devices to run LLM, and the speed is not too slow. The second is the Cosmopolitan project, which can compile programs at once to run on various operating systems and hardware.
Of course, this project will not be completed so quickly. Mozilla Foundation hopes that developers or users interested in LLM can participate: https://github.com/Mozilla-Ocho/llamafile