Apple's AI Trick: How a 20 Billion Parameter Model Fits on an iPhone
Local AI requires a lot of RAM. And that's precisely what's scarce on the iPhone. Apple therefore uses a special trick with its new AI model.
Apple's software chief Craig Federighi with an overview of the AI models used – the AFM Core Advanced model, which runs locally on the iPhone, can be seen in the bottom left.
(Image: Malte Kirchner / heise online)
Running an Artificial Intelligence model with 20 billion parameters on an iPhone – that initially sounds like a technical contradiction. Many smartphones simply don't have enough RAM for it, and Apple in particular has been frugal with equipping its devices with RAM in the past. Apple has nevertheless solved the problem for itself with a self-developed method that organizes the model fundamentally differently than the AI industry has done so far.
Normally, the entire model must reside in the device's fast RAM to function. A model with 20 billion parameters would quickly reach the limits of current smartphone hardware. Apple's previous on-device model therefore only had around three billion parameters – small enough to fit on the device, but correspondingly limited in its capabilities.
Flash memory solves RAM problem
(Image: Malte Kirchner / heise online)
AFM Core Advanced turns the classic principle on its head: the entire 20-billion-parameter model resides in flash memory, where photos, apps, and music are normally stored. Only the part actually needed for the current request is loaded into the fast RAM at any given time. The method was presented by Amar Subramanya, Apple's Vice President of AI, in a Tech Talk after the WWDC keynote, which Apple hosted at its headquarters in Cupertino.
The trick here: Instead of deciding anew with each individual word which parts of the model to activate – as common models do – AFM Core Advanced first analyzes the entire request and then selects the appropriate parameter subset once. This saves constant data transfer between flash and RAM – and thus time and battery power. This method is called Instruction-Following Pruning and was developed by Apple researchers.
A comparison: A doctor treating a patient doesn't need to access their entire medical knowledge simultaneously – only what is relevant to the specific case. That's precisely how AFM Core Advanced works.
Videos by heise
What that means in practice
The model is natively multimodal: it understands text, images, and audio simultaneously – and can output text and speech. Mike Rockwell, who leads the Siri project at Apple, said unequivocally at the Tech Talk: "We couldn't have built these new voices without this model." The new, much more natural-sounding Siri voices run on the device – without a cloud connection. The same applies to the improved dictation function.
Rockwell added what this means architecturally: "It's like we have a dedicated one-to-three-billion-parameter model just for voice – drawn from a much larger whole." And according to software chief Craig Federighi, the current use cases are "just the beginning." However, for certain AI functions, such as image generation, Apple plans usage limits that can be expanded via iCloud+ subscription. The model allows Apple to bring more and more performance to the device.
However, there's a catch: AFM Core Advanced is limited to the iPhone 17 Pro, iPhone Air, and high-performance Mac and iPad models. Which devices meet the full AI requirements of iOS 27 has been listed in detail by heise online. Those with older devices will continue to receive the smaller model. Apple Intelligence itself is only backward compatible to the iPhone 15 Pro. Apple is thus creating two AI classes within its own product line – and a possible reason to switch to new hardware. For developers, however, Apple offers the new cloud models free of charge under certain conditions.
(mki)
