DeepSeek's open-source DSpark technology had just been a week old before engineers "brought it into" Apple computers. This ported version called mlx-dspark boosted the generation speed of the Gemma-4 12B and Qwen3-4B models on Mac by 1.6 times and 1.4 times respectively, with output quality matching the original model byte by byte.
In the past, people thought Apple computers were slow at running large models because of different hardware architectures. Now, the top experts in the open-source community have written a "proprietary plugin" for Macs, not only boosting speed by 60% but also word for word. This proves that the performance ceiling of on-device AI is being continuously broken through software optimization.
| Apple computers speed up large model running by 60%
At the end of June, DeepSeek open-sourced DSpark technology. According to official data, it can accelerate server-side large model inference by 60% to 85%. However, at that time, this technology was only compatible with data center GPUs, and Apple users were still "eye-catching" and unusable.
Now, the situation has changed. Engineer Rahim developed mlx-dspark, the first native version of DSpark technology on Apple chips.
The acceleration logic of "small models assisting": DSpark's principle is essentially to give the main large models a "little assistant." The small model first guesses several candidate words in one go, then the large model checks them all at once: accept the correct ones directly, and type back the wrong ones to reguess.
However, the cost of this verification mechanism is different between data centers and Apple computers.
Data center GPUs: Like a "chartered car," no matter how many people are in the car, the price is fixed. Checking a few words rarely adds time costs.
Apple's chip is more like a "metered taxi"—the more candidate words you check, the longer it takes.
Rahim calculated the results after testing: on Apple chips, each additional word (token) verifying costs about 14 milliseconds. Based on this cost model, he concluded that the speed ceiling for Apple chips is about 2.2 times.
To achieve this, Rahim moved the small model to a Mac computer, rebuilt the verification process using Apple's own MLX framework, and compressed model weighting to 4-bit.
The real-world results on the M4 Pro chip are very impressive: the Gemma-4 12B model generation speed increased from the original 18.4 tok/s to about 30 tok/s, an increase of approximately 1.6 times. Qwen3-4B model: generation speed increased from 52.9 tok/s to about 73 tok/s, an increase of about 1.4 times.
｜移植也绝不“偷工减料”：输出精度做到逐字节对齐
很多把大模型搬到本地电脑上的版本，为了降低实现难度，通常只支持“贪婪解码”（也就是每一步都无脑挑概率最高的那个词）。但Rahim没有走这条捷径，他硬是把DSpark论文里更复杂的“温度采样”方法也完美实现了出来。
经过他自己反复核对，这套移植版跑出来的输出，和原生大模型在相同参数下给出的结果严格一致，做到了逐字节对齐，没有任何精度损失。
为了做到“又快又好”，Rahim在背后踩了不少坑，并摸索出了一套最优解：
小助手（草稿模型）要“轻量”：他把负责打前站猜词的小模型进行了4-bit量化压缩，体积仅有1.8GB，塞进内存毫无压力，而且跑起来依然无损。 大模型（目标模型）要“对口”：如果大模型用的是没经过指令微调的基础版，小助手猜对的概率只有47%；换成对应的指令微调版后，猜对的比例直接飙升到82%。 Accuracy selection must be "settled": He once tried switching large models to the higher-precision BF16, but found that the extra time spent far outweighed the benefits of improved accuracy, and actually slowed things down. Therefore, maintaining 8-bit accuracy for large models is the optimal solution for Apple chips.
After this series of fine-tunings, DSpark not only achieved a significant acceleration but also accurately reproduced the 16% to 18% acceptance rate mentioned in the paper on Apple devices.
|DFlash has joined the fray: code tasks are speeding up
After Rahim's tweet was posted, Jian Chen, one of the authors of the DFlash paper, left a comment hoping he could also test their team's model.
DFlash is a brand-new speculative decoding solution released by z-lab in May this year (its team leader, Zhijian Liu, is also a research scientist at NVIDIA). Its approach is completely different from DSpark: DSpark "guesses" the next word step by step with contextual dependencies, while DFlash uses parallel "block diffusion" technology to denoise and generate a whole block (16) token at once.
Rahim responded quickly, using a ready-made script to connect DFlash to the same Mac and had a "head-to-head" test with DSpark. The test results showed a clear "subject bias" phenomenon:
Code and math tasks (DFlash wins): DFlash's advantages are evident in these logically disciplined and highly predictable tasks. Of the 16 tokens generated at once, an average of 5.95 to 6.20 tokens are recognized by large models (i.e., extremely high acceptance lengths), with a generation speed of about 36 tok/s—about 2.1 times faster—directly outperforming DSpark. Open chat scenario (DSpark wins): In everyday chats with highly random content, DFlash's weaknesses are exposed. Because it pops up a whole word at once, the latter words often lack the previous dependencies, resulting in large models frequently not recognizing them during "review," significantly reducing actual acceptance length, and wiping out the speed advantage.
At this point, DSpark's built-in "Markov Head" mechanic comes into play. It specifically supplements the front-and-back dependencies for words generated in parallel, making candidate words more coherent. Therefore, in chat scenarios, DSpark actually runs faster than DFlash.
Kingtech Perspective | Edge-side inference is accelerating into a "flourishing" phase
In the past, the performance ceiling of local large models was considered entirely determined by chip manufacturing processes and memory. But the success of mlx-dspark proves that through targeted speculative decoding and quantitative strategies, there is still huge potential to tap into Apple's chip computing power.
With the leap in Mac-native large model experiences, inference acceleration tools, quantization algorithms, and model adaptation frameworks for specific hardware architectures (such as Apple Silicon) will become a new blue ocean for AI infrastructure investment.
Focusing on the "large-scale model collaboration" industry chain: the implementation of DSpark and DFlash has verified the feasibility of the "draft model + target model" architecture on consumer-grade devices. In the future, small-parameter draft models designed specifically for the device side, along with intelligent scheduling engines supporting dynamic block lengths, will see broad market demand.
Whoever can first achieve a "cloud-level" inference experience on consumer-grade devices will gain the upper hand in the battle for on-device AI entry points. For investors, focusing on software innovations that can "loosen" hardware is the optimal way to seize this round of industry dividends.
