Apple's AFM 3 Moves Model Weights Off DRAM — and Changes the On-Device AI Calculus

A new flash-based architecture lets a 20-billion-parameter model run locally on Apple silicon. The memory wall just moved. The compliance questions haven't been answered yet.

Written by Lena Armitage · Bureau Tech · Jun 13, 2026

The constraint every on-device AI developer runs into

On-device AI models have stayed small for a straightforward reason: the entire weight set has to fit in DRAM. That caps practical parameter counts well below what server-side deployments use, and it has forced enterprise architects evaluating agentic workloads into an uncomfortable binary — capable cloud-dependent models, or limited local ones.

Apple's AFM 3 Core Advanced, announced at WWDC26, is a direct attempt to break that constraint. The model stores its full 20-billion-parameter weight set in NAND flash (the same storage used for files) rather than active memory, and loads only the parameters a given prompt requires into DRAM at inference time.

"You can't put 20B parameters in RAM at any reasonable precision," Awni Hannun, a researcher at Anthropic and former Apple research scientist, wrote on X. "To make it work they are using pretty exotic architecture by today's standards."

How the routing mechanism actually works

AFM 3 Core Advanced is a Mixture of Experts (MoE) model — an architecture where different subsets of parameters, called experts, handle different types of tasks. In a standard MoE, a router selects different experts for every token generated. That would require continuous weight movement between flash and DRAM at inference speed, which NAND-to-DRAM bandwidth cannot support.

Apple's solution is to route once per prompt instead. A small model predicts which experts the query requires, loads that fixed set into DRAM alongside always-active shared experts, and generates all tokens from that same configuration. "The key distinction from a typical MoE is that you do this once per query and then generate all the tokens with the same experts," Hannun wrote.

The active parameter count scales with task complexity — roughly 1 billion parameters for simpler operations, up to 4 billion for harder ones — all drawn from the 20-billion-parameter pool in flash. Apple calls the pruning technique used to develop this Instruction-Following Pruning (IFP).

What Apple hasn't disclosed

The architecture paper is detailed on the memory design. It is less forthcoming on the metrics that determine whether the design is viable in production.

Marco Abis, who is building Ziraph, a profiler for local AI on Apple silicon, noted on X that Apple's profiling tools expose timing but not energy consumption, memory bandwidth, or thermal behavior. "A notable gap," he wrote, "given those decide most of on-device performance."

More consequentially for enterprise deployments: Apple has not documented when an on-device request transparently offloads to the cloud, or whether that routing decision is visible to the developer or the user. For organizations in regulated industries that need to document where inference runs, that is not an abstract concern — it is a compliance problem.

Apple has indicated a full technical report with benchmarks is coming later this summer. Until then, the deployment picture is incomplete.

The broader AFM 3 family and its dependencies

AFM 3 Core Advanced is one of five models in the AFM 3 family, developed in collaboration with Google. Two models run on-device; three run server-side within Apple's Private Cloud Compute boundary. The server-side agentic model, AFM 3 Cloud Pro, is designed for complex reasoning and tool use and runs on Nvidia GPUs in Google Cloud.

Private Cloud Compute provides a data-privacy guarantee. It does not eliminate the Google Cloud infrastructure dependency for server-side inference — a distinction worth keeping clear when evaluating what "private" means in practice.

What changes for enterprise architects

The DRAM ceiling for on-device agents just moved. Enterprises evaluating agentic workloads that need to run without a cloud round-trip now have a 20-billion-parameter local option that did not exist before WWDC26. Whether it is deployable at scale — given the unanswered questions about thermal behavior, offload transparency, and compliance visibility — depends on answers Apple has not yet published.

Key takeaways

AFM 3 Core Advanced is a 20-billion-parameter on-device model that stores weights in NAND flash, not DRAM — a meaningful departure from how every practical on-device model has worked until now.
Expert routing happens once per prompt, not per token, which is what makes the slow NAND-to-DRAM bandwidth viable at inference time.
Active parameter count scales between 1B and 4B depending on task complexity, drawn from the full 20B pool in flash.
Apple has not publicly specified when an on-device request transparently offloads to the cloud, or whether that routing is visible to developers — a direct problem for regulated industries that need to document where inference runs.
The server-side agentic tier, AFM 3 Cloud Pro, runs on Nvidia GPUs in Google Cloud. Private Cloud Compute covers data privacy; it does not eliminate the Google Cloud dependency.

FAQ

Why have on-device AI models been limited in size until now?

Because the entire model weight set has to fit in DRAM during inference. Consumer devices have far less DRAM than server hardware, which caps practical parameter counts well below what cloud-deployed models use. Apple's AFM 3 Core Advanced works around this by storing weights in NAND flash and loading only the relevant subset into DRAM per prompt.

What is a Mixture of Experts model, and why does it matter here?

A Mixture of Experts (MoE) model divides its parameters into specialized subsets called experts and activates only a fraction of them for any given input. This lets a model have a large total parameter count while using far fewer parameters per inference. AFM 3 Core Advanced uses MoE architecture, but routes once per prompt rather than per token — a key modification that makes flash-based weight storage viable.

Does Private Cloud Compute mean Apple's server-side models don't touch Google's infrastructure?

No. AFM 3 Cloud Pro, the server-side agentic model, runs on Nvidia GPUs in Google Cloud. Private Cloud Compute is Apple's data-privacy framework — it governs how data is handled, not which physical infrastructure the compute runs on. The Google Cloud dependency remains.

What compliance concerns does the current documentation leave unresolved?

Apple has not publicly specified when an on-device inference request is transparently offloaded to the cloud, or whether that routing decision is visible to the developer or the end user. For regulated industries — healthcare, finance, legal — knowing where inference runs is often a documentation requirement, not a preference. That information is not currently in Apple's published materials.

When will Apple publish full benchmarks for AFM 3?

Apple has indicated a full technical report with benchmarks is planned for later in summer 2026. Until that report is available, independent evaluation of the model's performance claims is not possible.

Citations

On-device AI agents hit a hard memory limit. Apple's new architecture routes around it.AFM 3 Core Advanced stores its full 20-billion-parameter weight set in NAND flash rather than DRAM, routing once per prompt to select which experts to load into active memory.
VentureBeat — Apple AFM 3 architecture coverage (via FeedBurner)Apple's AFM 3 family spans five models developed in collaboration with Google, with server-side models including AFM 3 Cloud Pro running on Nvidia GPUs in Google Cloud.
On-device AI agents hit a hard memory limit. Apple's new architecture routes around it.Marco Abis noted that Apple's profiling tools expose timing but not energy, memory bandwidth, or thermal metrics — and that Apple's documentation does not specify when an on-device request offloads to the cloud.
On-device AI agents hit a hard memory limit. Apple's new architecture routes around it.Awni Hannun, researcher at Anthropic and former Apple research scientist, described the architecture as 'pretty exotic by today's standards' and explained that a small model predicts which experts to load from NAND into RAM per query.