KAN (Kolmogorov-Arnold Network)

The Kolmogorov-Arnold Network, or KAN, is a neural architecture that replaces the fixed activation functions and learned scalar weights of a standard multilayer perceptron with learned univariate functions placed on the edges of the network. Instead of computing y = σ(Σ w·x + b) at each neuron, a KAN node simply sums its incoming edge activations, each of which is a small, separately trainable function of a single input.

The architecture takes its name from the Kolmogorov-Arnold representation theorem, which states that any multivariate continuous function on a bounded domain can be written as a finite composition of sums and univariate continuous functions. KANs realize this decomposition as a network: each layer sums per-edge learned functions, and stacking layers gives a depth-wise composition. The original formulation parameterizes each edge function as a linear combination of B-spline basis functions on a small fixed domain (typically [−1, 1]), with the spline coefficients trained by gradient descent. The bounded, univariate, smooth structure makes each edge function easy to learn, easy to visualize, and cheap to evaluate.

On standard benchmarks KANs have been shown to match MLPs with substantially fewer parameters and to expose more interpretable structure — individual edge functions often correspond to recognizable features of the target. The cost is that the spline parameterization adds a non-trivial per-edge memory footprint and that training can be slower than an equivalently sized MLP. The architecture is most attractive in regimes where parameter efficiency, interpretability, or hardware-friendly evaluation matter most — for instance, fitting scientific data, or deploying to fixed-function accelerators.

Whether KANs' empirical advantages hold at the scale of modern foundation models is unsettled: most published comparisons are at small parameter counts. The choice of basis (B-splines, Chebyshev polynomials, Fourier bases, wavelets, or small MLPs on a single input) and how to scale the architecture to deep, wide networks remain active research questions. Recent work has also explored variants where edge functions are themselves neural networks or where the architecture is co-designed with hardware like FPGAs, exploiting the fact that a univariate bounded function is a natural lookup-table target.