Neural Turing Machine
Combines a neural network with readable and writable external memory.
A neural Turing machine is a model that learns to use an external memory while solving a task. It pairs a neural controller with a memory matrix that can be read from and written to. The goal is to let a model learn algorithm-like behavior from examples rather than from hand-written rules.
The controller emits differentiable read and write operations over memory locations. It can address memory by content, by position, or by shifting focus from a previous location. Because these operations are differentiable, the whole system can be trained with gradient descent on tasks such as copying, sorting, or associative recall.
The appeal is that explicit memory can represent variables, lists, and intermediate state more directly than a fixed hidden vector. The downside is that training can be fragile, memory access can be inefficient, and learned algorithms may fail outside the lengths seen during training. Later memory-augmented models and attention mechanisms inherited many of its ideas while often simplifying the interface.
Open questions remain about whether explicit differentiable memory is necessary in an era of long-context transformers and retrieval systems. Discrete memory operations may generalize better, but are harder to train end to end. The deeper issue is how to make neural systems reliably learn reusable procedures rather than only smooth approximations of the training distribution.