Abstract:
Recently, developer Cactus Compute released a lightweight AI model called Needle 2.
The model is only 14MB in size and is deeply optimized for the processor environment of Raspberry Pi 5. It can convert natural language instructions into preset function calls in real time without the need for an AI acceleration expansion board.
According to the Raspberry Pi official blog, Needle 2 is not a traditional general-purpose chat robot, but a dedicated model focused on device-side command execution. When the user inputs to light up the LED light, the system only takes 78 milliseconds to complete the entire process from recognition to execution, achieving almost instant response.
In terms of performance and resource usage, Needle 2 also demonstrates extreme efficiency. In the actual measurement of Raspberry Pi 5 (8GB memory version), the memory occupied by this model when running is only about 28MB. Even if the complete Python demonstration program is run, the total memory consumption is only 46.4MB.
This lightweight design enables the model to be completely separated from the cloud API and network environment, and achieve purely local intelligent control at the edge.
In terms of hardware features and interface calls, Needle 2 is deeply bound to the underlying hardware through the Python annotation system. Developers only need to add specific tags to the existing function code, and the model can automatically capture the function name, description, and parameter types, and build a calling specification.
For example, the response time of an instruction to query the processor temperature is 149 milliseconds, and the system will accurately return the Celsius value. The entire process is completely completed within the local processor.
However, it should be noted that the model is very restrained in logical judgment. For irrelevant questions that cannot match the preset function (such as "Where is the capital of France?"), Needle 2 will return a null value within 92 milliseconds instead of forcibly answering.
Currently, the model weights of Needle 2 have been released on the Hugging Face platform, and the relevant code is also open source on GitHub following the Apache 2.0 protocol.
Comments