DQN (Deep Q-Networks)
Reinforcement learning method combining Q-learning with deep neural networks for complex environments.
Deep Q-Networks (DQN) are a class of reinforcement learning algorithms that use deep neural networks to approximate the Q-value function, a measure of how valuable it is to take a particular action from a given state. Classical Q-learning maintains a lookup table mapping every state-action pair to an expected cumulative reward, but this approach becomes computationally intractable in environments with high-dimensional inputs like raw pixel images. DQN addresses this limitation by training a convolutional neural network to generalize across similar states, compressing the Q-table into a learned function that scales to complex, real-world-like problems.
Two innovations made DQN training stable enough to work in practice. The first is experience replay, in which the agent stores past transitions in a memory buffer and samples random mini-batches during training, breaking the temporal correlations that would otherwise destabilize gradient updates. The second is a target network, a periodically frozen copy of the main network used to compute training targets, preventing the feedback loop that arises when both the predictions and the targets shift simultaneously. Together, these techniques turned a theoretically appealing but practically fragile idea into a reliable learning algorithm.
DQN drew widespread attention after DeepMind demonstrated it learning to play dozens of Atari 2600 games directly from pixel inputs, achieving superhuman performance on several titles without any game-specific engineering. A single architecture and set of hyperparameters was applied uniformly across games with very different dynamics, which suggested that DQN had learned general strategies rather than narrow heuristics.
DQN's influence extends beyond Atari. It established a template for deep reinforcement learning research and produced a family of improvements, including Double DQN, Dueling DQN, Prioritized Experience Replay, and Rainbow, each addressing specific weaknesses in the original formulation. DQN is a foundational reference point in RL, taught in virtually every modern course on the subject and used as a baseline against which newer algorithms are routinely benchmarked.
Sources
- Playing Atari with Deep Reinforcement Learning
arXiv · Jan 1, 2013
Research this in Signals
Signals turns a topic into a sourced research record you can inspect and rerun. Your first scan is free, and this one starts with DQN (Deep Q-Networks) already loaded, so edit it or scan as is.
