OpenAI Launches Preparedness Team to Address AI Safety and Risk Management
OpenAI is taking further steps in fulfilling its mission of building safe and beneficial artificial general intelligence (AGI). The organization recognizes the importance of addressing a wide range of safety risks associated with AI, from the current state of technology to the potential challenges posed by superintelligent AI systems in the future.
In July, OpenAI, along with other leading AI research labs, made voluntary commitments aimed at enhancing safety, security, and trust in artificial intelligence. These commitments encompass various risk areas, with a particular focus on frontier risks highlighted during the UK AI Safety Summit.
OpenAI’s commitment to preparedness involves developing strategies to handle the increasing risks associated with frontier AI models, which are expected to surpass the capabilities of existing models. While these advanced AI models hold great potential for humanity, they also present substantial risks.
The key questions OpenAI aims to address include:
- How dangerous can frontier AI systems become when misused, both presently and in the future?
- What measures can be implemented to establish a robust framework for monitoring, evaluating, predicting, and safeguarding against the potentially harmful capabilities of frontier AI systems?
- In the event of unauthorized access to the weights of frontier AI models, how might malicious actors exploit this?
To mitigate these risks and ensure the safety of highly capable AI systems, OpenAI is launching the Preparedness team. This team, led by Aleksander Madry, will play a crucial role in connecting capability assessment, evaluations, and internal red teaming for frontier AI models, ranging from near-future models to those with AGI-level capabilities.
The Preparedness team’s responsibilities will include monitoring, evaluating, forecasting, and protecting against catastrophic risks in several critical areas, such as individualized persuasion, cybersecurity, threats related to chemical, biological, radiological, and nuclear (CBRN) risks, as well as issues concerning autonomous replication and adaptation (ARA).
Furthermore, the Preparedness team is tasked with developing and maintaining a Risk-Informed Development Policy (RDP). This policy will outline OpenAI’s approach to conducting rigorous evaluations of frontier model capabilities, implementing a spectrum of protective actions, and establishing a governance structure for accountability and oversight throughout the development process.
OpenAI’s ongoing commitment to risk mitigation and preparedness underscores its dedication to ensuring the safety and responsible development of advanced AI systems, both before and after deployment.
Here is the link: https://openai.com/blog/frontier-risk-and-preparedness
