Taobao mnn team releases 0.3B mobile LLM, achieving 350 Tokens/s on iPhone 18 pro

The MNN team at Taobao has unveiled MNN-Pocket-0.3B, a lightweight language model designed for mobile applications, balancing performance, speed, and power consumption under constrained resources. This model, featuring approximately 300 million parameters, was engineered to achieve performance comparable to 8B-level models while prioritizing instruction following for both Chinese and English. On a real iPhone 18 Pro, the model demonstrated an average decoding speed of about 350 tokens/s when running entirely offline using the MNN inference engine. Key technical specifications include a peak memory usage below 300MB and a decoding throughput exceeding 200 tokens/s under specified configurations. The deployment file size for the INT4 MNN version is only 173MB. Comparative testing on an iPad Pro M5 showed MNN-Pocket-0.3B had a 49% smaller deployment size and 47% lower peak memory increase compared to Qwen3-0.6B (340MB/427MB increase, respectively). In terms of efficiency, the model's incremental energy consumption during typical inference is targeted to be less than one ten-thousandth of the battery's energy per inference. Furthermore, the team reported an average incremental energy consumption of about 2.7J per round on a OnePlus 13 (Snapdragon 8 Gen 2) under a 1024 input/128 output load. The model achieved a Chinese instruction-following IFEval-zh score of 74.59, the highest among comparable models tested.