The AI race has become one of the defining leadership and security challenges of our time.
Recent disclosures about AI agents ‘going rogue’ illustrate that AI systems can autonomously cross boundaries and exceed the mandates set by their designers. In documented cases, human design and governance choices, together with security weaknesses and implementation failures, materially shaped the conditions enabling those incidents. Humans still largely determine access and safeguards, which in turn shape how much harm can result.
A prevalent assumption in the AI debate is that speed is imposed by competition: slow down too much and a rival — above all China — may overtake you. This logic appears broadly in industry rhetoric and government policy, even among leading advocates of AI safety.
In his recent call to ‘pace the frontier’, Dario Amodei argues for giving safety more time to catch up with capability and outlines several ways to do so. Yet he accepts that democracies’ room to slow development is limited by their lead over China.
Race pressure narrows focus
The pressure of any race naturally creates a mindset that keeps immediate focus on achieving ever greater capability and speed. Organisations adopting AI face similar pressure to move quickly, fearing they may fall behind their competitors.
In both cases, the urgency to keep pace risks becoming a purpose onto itself. Engineering and measures required to make AI systems work reliably — including safety engineering — is easily seen as slowing development, risking national-security disadvantage or loss of competitive edge.
This is where leadership logic risks becoming flawed. Compare it with a driver in a racing car deciding that keeping his foot on the accelerator is the only way to win, and using the steering wheel or brakes is likely to cost him the race.
Competitive pressure constrains choices, but does not remove the need for human control. Anthropic itself shows how frontier-AI companies decide what to develop, what authority to give their systems, what to release and when safety concerns warrant hitting the brake. Governments still decide what conditions to set around deployment.
China’s behaviour makes this framing even more debatable. Frequently invoked as the reason Western developers cannot afford restraint, it is pursuing rapid AI development while actively developing controls around agent permissions, human decision authority, anomaly detection, intervention and shutdown. While the purposes of those controls include state-security and political motivations, they suggest that China does not view controllability as an impediment to competing successfully.
During his White House visit in September, President Xi made that position explicit, stating that AI development should remain under human control.
Redefining the winner
That raises a central question: how do we define the winner of the AI race?
Capability tells us what a system can do. Controllability by humans determines how much of that capability is safe and feasible to use. If more capable systems repeatedly breach human-defined boundaries and pose real dangers, simply increasing raw capability stops being a meaningful goal.
Safety is part of engineering
In his 23 September interview with The New York Times’ Ezra Klein, Jensen Huang made a key counterpoint: safety is part of engineering, not an impediment to technological progress. Klein proposed thinking of safety and alignment as capability expansion and said “An unsafe technology is not an advancing technology.” Huang concurred and compared AI safety engineering to Nvidia's investments in verification and reliability engineering.
Military aviation offers a striking analogy. During the First World War, aircraft warfare developed at extraordinary speed – and at appalling odds of survival for pilots.
One early engineering problem now sounds absurd: the most effective position for a fighter’s machine gun meant firing through its own propeller. French aviation pioneer Roland Garros estimated that 7–10 per cent of bullets would hit a blade. This led to a race to redesign the system. Within months, Anthony Fokker’s team developed a synchronization mechanism for Germany that prevented the gun from firing when a blade crossed its line of fire.
None of the combatant powers concluded that firing into your own propeller would permanently remain the unavoidable price of staying competitive. On the contrary: once synchronization gave Germany a clear combat advantage, the Allies raced to catch up.
The Andon principle in AI
High-performance manufacturing views containing faults before they multiply as sound cost and risk management. Toyota’s Jidoka and Andon principles provide a useful model: Jidoka builds the ability to detect abnormalities and stop production when they occur; Andon makes problems visible so people can intervene.
On 29 September, Anthropic, OpenAI, Google, Meta, xAI and Nvidia signed the White House Accord on Super Intelligence, committing to four layers of controls and audits around frontier AI. While this accord is voluntary and leaves much oversight in company hands, it is an important step towards treating controllability as part of what we determine is usable capability.
Here the real problem begins
Control challenges extend beyond engineering. Political differences complicate international information-sharing. Unlike traditional state-led arms races, AI puts increasingly powerful capabilities within reach of individuals and small groups with intentions ranging from Good, via Bad, straight to Ugly. This decentralisation makes control harder.
Preventing highly capable AI from crossing human defined boundaries also presents technical challenges. Examples include distinguishing legitimate instructions from malicious ones hidden in information agents process; building monitoring systems that provide reliable signals and are hard to evade; and restricting permissions or isolating agents from external systems without limiting key functionality.
Humans need to retain authority over which AI loops may run without them and how: what access systems may have, which decisions they may take autonomously, what triggers containment, and who may stop or restart them. Yet human approval cannot keep pace with every action at computational speed; people can become overly reliant on AI output; and human stop mechanisms are effective only in cultures where people can use them without risking their livelihood, even when a stop proves unnecessary.
Effective control therefore depends on multiple overlapping safeguards working together.
Human agency at the core
The large language model I’ve been using outperforms me on processing speed — including in researching this article. It can find sources faster, test arguments and help me improve prose.
This shifts rather than removes my human capacity bottleneck: I still need to verify information, prioritise what matters and edit text to represent my voice. AI doesn’t decide what question needs asking or how the argument should be focused. Nor do I intend to surrender that authority by letting it autonomously post unchecked text.
The same distinction applies at scale. AI can extend human capacity in some areas while creating new bottlenecks in others. It shifts where human effort is needed. It doesn’t remove human leadership responsibility or human limitations.
“And the winner is…”
As AI is increasingly used for financial transactions, coding, infrastructure and consequential decisions, the most valuable AI is the kind that can be trusted to operate safely within human parameters. This means the winner of the AI race is whoever can build AI that can be used most widely and effectively while reliably remaining within our control.
Defining the boundaries of control — and retaining the authority and ability to enforce them — will remain our job.
Comments