Artificial intelligence governance is shifting from the question of whether regulation is necessary to the more immediate test of whether regulation can keep pace with technological expansion. In early and mid-September, a series of warnings from U.S. political figures, the United Nations human-rights system, and within the AI industry itself indicated that debates over artificial intelligence are no longer confined to innovation and commercial competition. They now extend to employment, democratic institutions, human-rights protections, and global security.
On September 7, Volker Türk, the United Nations High Commissioner for Human Rights, told the 63rd session of the UN Human Rights Council that advanced AI could pose an “existential” risk to humanity. He called on governments and leading AI companies to accelerate cooperation, establish robust safeguards before risks become unmanageable, and draw clear red lines around unacceptable uses of the technology.
On September 13, several media outlets, citing The New York Times, reported that former U.S. President Barack Obama had warned at a private fundraising event that artificial intelligence could be highly “dangerous” if poorly governed. He urged Democrats to place AI at the center of their political agenda and to develop a clear policy programme addressing both its economic consequences and safety risks.At roughly the same time, calls to slow the race to develop increasingly capable AI systems regained prominence. Jakub Pachocki, OpenAI’s chief scientist, said on September 7 that society was not adequately prepared for the continuing rapid rise of machine intelligence. He urged extreme caution until common safety standards are in place. These concerns converge on a single problem: while model capabilities, computing investment, and commercial deployment are advancing rapidly, systems for testing, auditing, accountability, and international coordination remain insufficient.
Three Warnings, One Governance Gap
The three sets of concerns arise from different perspectives. Political leaders focus on how AI may disrupt employment and economic distribution, amplify disinformation, and affect democratic processes. The United Nations human-rights framework is concerned with algorithmic discrimination, privacy violations, mass surveillance, and the concentration of technological power. Industry researchers and leaders, meanwhile, are more directly focused on the risks of loss of control, misuse, and potentially catastrophic consequences associated with highly autonomous systems.
Yet all three point to the same governance gap: AI capabilities are developing faster than public rules, social adaptation, and safety-validation mechanisms.
That gap is particularly apparent with the emergence of highly autonomous AI. Unlike conventional software, systems that can plan tasks, use external tools, access databases, write and execute code, and act across digital environments do more than merely generate answers. They can take action. If such systems are maliciously manipulated, improperly configured, or behave unexpectedly in complex settings, the consequences may extend beyond inaccurate outputs to cyberattacks, financial fraud, privacy breaches, or failures involving critical systems.
AI governance should therefore extend beyond questions of content accuracy and data compliance. It must also address whether systems possess dangerous capabilities, whether they may be connected to real-world systems, whether abnormal behaviour can be stopped promptly, and whether responsibility can be clearly traced.
Slowing Development Means Setting Thresholds
Calls to “slow AI development” are often misconstrued as opposition to innovation. That is not the central argument. The issue is not whether AI research should continue, but whether model autonomy and deployment should expand without limit when safety conditions remain inadequate.
Technological progress does not automatically translate into social progress. Innovation becomes sustainable only when risks can be identified, assessed, and controlled. Aviation, pharmaceuticals, and financial services have all evolved alongside testing, certification, auditing, and accountability regimes. Such safeguards have not prevented innovation; rather, they have established the baseline trust necessary for technologies to be widely adopted.
AI should be treated similarly. Regulation should remain proportionate for lower-risk tools, such as translation, office assistance, and information organisation, so that excessive compliance burdens do not stifle innovation. However, systems used in healthcare, criminal justice, credit decisions, recruitment, education, social welfare, cybersecurity, and critical infrastructure should be subject to substantially more stringent safeguards. Regulation should therefore be based not simply on a company’s size or a technology label, but on a system’s capabilities, intended use, and potential harms.
From Statements to Enforceable Rules
What is now needed is not another layer of abstract ethical principles, but institutional arrangements that can be tested, enforced, and used to assign responsibility.
First, frontier models with a high degree of autonomy, powerful tool-use capabilities, or significant dual-use risks should be subject to tiered oversight. This could include development registration, pre-deployment safety assessments, and delayed release or use restrictions when risks cannot be adequately controlled.
Second, independent safety-evaluation systems should be established. Internal testing remains essential, but it cannot fully substitute for external auditing. High-risk models should undergo third-party red-team testing, including assessments of their capacity to assist cyberattacks, engage in deceptive conduct, operate autonomously, disseminate dangerous knowledge, or circumvent safeguards.
Third, incident-reporting and accountability mechanisms must be strengthened. Companies should be required to report serious vulnerabilities, anomalous behaviour, data breaches, and large-scale misuse promptly. Systems deployed in public services or critical sectors should retain adequate operational logs, version histories, and human-override mechanisms.
Finally, the international community should establish minimum safety red lines in areas where consensus is most attainable. These should include restrictions on lethal-weapons decisions made without meaningful human control; safeguards against indiscriminate mass biometric surveillance; measures to prevent AI-enabled electoral manipulation; and prohibitions on connecting highly autonomous systems to critical infrastructure without rigorous safety testing.
The Window for Governance Is Narrowing
From the UN human-rights chief’s urgent appeal on September 7 to Obama’s call on September 13 for a clear Democratic AI agenda, the message is straightforward: AI governance can no longer remain at the level of general principles or voluntary corporate commitments.
The challenge is to avoid two errors at once: restricting innovation so severely that societies forgo its benefits, while also allowing competitive pressures to shift safety costs onto the public. Sound policy is not a simplistic choice between acceleration and suspension. It is the creation of clear boundaries between innovation incentives and public safety.
If rules continue to lag behind expanding capabilities, societies will be forced to respond after harm has occurred. Once high-risk AI systems are embedded in critical infrastructure, public decision-making, or large-scale information ecosystems, the costs of correction may far exceed the costs of prevention. Prudent AI governance is not an attempt to constrain the future; it is an effort to preserve society’s ability to shape it.
From: Sok Sovan
