Capability control is a critical aspect of AI safety and governance, aiming to design and manage AI systems in ways that they do not exceed or misuse their intended capabilities. This involves developing technical measures, ethical guidelines, and governance frameworks to limit the autonomy of AI systems, ensuring they cannot take actions beyond their purpose or that could lead to unintended consequences. Capability control is especially relevant in the context of advanced AI and autonomous systems, where the potential for actions that developers did not foresee increases. It includes mechanisms like constraint satisfaction, ethical alignment, and fail-safes that ensure AI actions remain aligned with human values and safety requirements.
The concept of capability control has gained prominence alongside the rapid advancement of AI technologies, particularly from the early 21st century as researchers and policymakers began to recognize the potential risks associated with highly autonomous systems. The precise origins of the term are difficult to pinpoint, but discussions around limiting the power and capabilities of AI systems to prevent undesired outcomes have been part of the AI ethics discourse for several decades.
While specific individuals directly associated with the term "capability control" are not well-documented, organizations like the Future of Humanity Institute, the Center for the Study of Existential Risk, and the Machine Intelligence Research Institute have been instrumental in advancing research and dialogue on this topic. Scholars such as Nick Bostrom and Eliezer Yudkowsky have significantly contributed to the broader discussions on AI safety and ethics, which include aspects of capability control.