GRPO
Grouped Regularized Policy Optimization

Grouped Regularized Policy Optimization (GRPO) augments standard policy‑optimization objectives (e.g., policy gradient, TRPO, PPO) with explicit grouped regularizers or group‑aware constraints that tie together subsets of parameters, actions, or policy components. Practically, GRPO inserts group‑wise penalties (for example group‑L1/L2, L2,1 norms, or blockwise KL/entropy constraints) into the surrogate objective or enforces proximal updates that act across pre‑defined or learned groups; this encourages sparsity, structured parameter sharing, or coherent updates for related actions/state partitions and reduces variance and overfitting. From a theoretical perspective, GRPO leverages the bias–variance/regularization tradeoff to improve sample efficiency and robustness, and it can be viewed as combining ideas from proximal policy optimization, natural‑gradient methods, and structured regularization (group lasso) to obtain stable, interpretable policy updates—especially valuable in multi‑task, hierarchical, or federated RL (Reinforcement Learning) scenarios where grouped structure is natural.

The explicit label "Grouped Regularized Policy Optimization" does not have a single canonical origin; its component techniques trace to earlier work such as REINFORCE (policy gradients), TRPO/PPO (stable surrogate objectives and proximal updates), and group‑regularization literature (e.g., group lasso). In practice, variants of GRPO began appearing in the early 2020s in research on multi‑task and federated RL, and the approach gained broader attention through the mid‑2020s as researchers sought stable, sample‑efficient methods that exploit known groupings (actions, skills, sensors, agents) or induce block‑sparse representations for scalability and interpretability.

Foundational contributors are therefore a mix of policy‑optimization and structured‑regularization researchers: Williams (REINFORCE) and Sutton/Barto for early policy‑gradient theory; Schulman et al. (TRPO, PPO) for modern stable surrogate objectives; researchers on structured sparsity such as Yuan & Lin (group lasso) and Tibshirani for regularization foundations; and numerous contemporary RL groups (academia and industry) advancing multi‑task, hierarchical, and federated RL who have adapted grouped regularization ideas into policy optimization. Specific algorithmic instantiations and empirical validations are mainly found in papers and workshops from RL researchers integrating group‑wise penalties into surrogate objectives rather than a single seminal "GRPO" paper.