Anthropic and OpenAI diverge on who should pull the plug on risky AI

Publisher excerpt From CSO Online

OpenAI and Anthropic agree that frontier AI systems need stronger oversight, but they are proposing sharply different approaches to stopping models that pose serious risks: Anthropic wants courts empowered to suspend dangerous systems, while OpenAI argues developers should be required to maintain and use effective shutdown controls. The proposals appear in separate submissions to California Governor Gavin Newsom following his September 18 executive order directing state officials to consider additional safeguards for frontier AI systems. Both companies support independent safety assessments and expanded reporting of incidents involving AI systems that act beyond their intended permissions. “If the AG concludes, informed by the independent evaluation, that a frontier AI company’s models pose a near-term risk of catastrophic harm, they should be able to seek a court order to suspend a model’s deployment, internal use, or further development,” Anthropic wrote in its October 15 submission. OpenAI, in its October 20 letter, called for an amendment to California’s Senate Bill 53 (SB 53). “We support amending SB 53 to require frontier developers to maintain effective shutdown controls and an enforceable duty to use them when unsafe conditions arise, and no lesser intervention would adequately address the risk,” OpenAI wrote. Sanchit Vir Gogia, chief analyst at Greyhound Research, said the two approaches address complementary needs. “The operator must contain the unsafe activit…

Excerpt only, shown with attribution. This content belongs to the original publisher and has not been independently verified by CyberSecurity.pk.