AFMs (Analog Foundation Models)

Analog Foundation Models (AFMs) are foundation-scale AI systems architected or adapted to execute primarily on analog compute substrates — such as memristive crossbars, photonic processors, or neuromorphic circuits — rather than conventional digital accelerators. The core motivation is physical: analog hardware can perform matrix-vector multiplications, the dominant operation in neural networks, directly in memory using Ohm's law and Kirchhoff's current law, collapsing the costly data movement between memory and compute that bottlenecks digital systems. Photonic variants exploit light-speed interference for similar operations. The result, in principle, is inference at orders-of-magnitude lower energy and latency, making large-model deployment feasible at the edge, in mobile robotics, or within strict data-center power budgets.

Building AFMs requires deep co-design across hardware and software. Analog devices introduce challenges absent in digital systems: resistive drift, thermal noise, limited weight precision, and nonlinear device responses all degrade model accuracy if left unaddressed. Practitioners counter these with analog-aware training techniques — injecting device noise during optimization, applying hardware-specific quantization, and inserting learned calibration layers that compensate for systematic nonidealities. Architecture-level choices also matter: sparse activations, normalization schemes tolerant of analog variability, and hybrid digital-analog splits (where sensitive operations remain digital) are common design patterns. Compiler and mapping algorithms must further translate trained models onto physical crossbar or photonic topologies, respecting device constraints.

The significance of AFMs is twofold. First, they extend the foundation-model paradigm — large, generalizable, pretrained models — into hardware regimes previously inaccessible due to power or latency constraints. Second, they open new research frontiers in robust optimization, device-aware model compression, and co-designed toolchains that fundamentally reframe the tradeoff between model scale and hardware cost. Prototype demonstrations on memristive and photonic accelerators hosting increasingly large pretrained models, along with early on-chip fine-tuning results, have established analog approaches as a credible complement to purely digital ML infrastructure, driving growing academic and industry investment in the field.