Iterated Amplification
An AI training technique that enhances task performance by recursively improving a base agent with the help of human supervision and task decomposition.
Iterated amplification is a crucial technique in AI alignment and safety, aimed at increasing the capability of AI models through recursive training processes. It involves the augmentation of a base model by iteratively applying task decomposition, where complex tasks are broken down into simpler sub-tasks handled by previous iterations along with human input. The core idea is that by repeatedly amplifying the model's capabilities, the resulting model can tackle tasks that originally required human intelligence. Iterated amplification holds theoretical significance as it offers a systematic approach to refine and align AI systems, making them more robust and reliable across various applications while ensuring adherence to human-aligned objectives.
First conceptualized in the mid-2010s, iterated amplification gained attention as researchers sought reliable methods to ensure that increasingly powerful AI models remain aligned with human values and objectives.
Paul Christiano is a key figure in the development of iterated amplification, promoting the concept through his work at OpenAI and significantly advancing the dialogue on AI safety and alignment.
