Beware of Industry Leaders Demanding Regulation
A cabal comprised of some of the biggest players in the artificial intelligence industry has discovered a potentially lucrative application for human intelligence: persuading Washington to regulate the sector and hold down potential competitors.
Over the weekend, Anthropic chief executive Dario Amodei called for slowing the development of advanced AI, embedding outside safety evaluators inside leading laboratories, and establishing common standards with government help. Sam Altman and Elon Musk endorsed the push. There’s also been talk of antitrust waivers to permit competitors to talk to each other about the new rules of the AI road.
The warnings accompanying these proposals are alarming. Last week, a 27-year-old Anthropic researcher named Jacob Coxon announced his resignation, accusing the company and his former employer, OpenAI, of “gambling with our lives.” He said people building AI privately believe it could wipe out humanity before the decade ends.
A researcher still inside Anthropic publicly backed him. “We really do earnestly believe AI could kill all humans,” wrote Evan Hubinger, the company’s alignment science lead. He put his personal estimate of that risk at greater than 10 percent within the next decade and acknowledged that researchers lack a solution for reliably controlling superintelligent systems. That’s not a risk right now—and this will be important later—but a prospective danger from models that might be developed in the future.
Amodei’s own warning includes AI agents conducting unauthorized cyberattacks and the possibility that more powerful systems could escape human control. His prescription would slow development to give safety research time to catch up.
Obamanomics: The Forgotten Fraud
Rahm Emanuel was shocked—shocked—that captains of industry would ask to be regulated.
This raises substantial doubts about whether Emanuel has experienced some sort of cognitive decline or if the Obama administration just shut him out of all the important conversations. In reality, the willingness of business executives to seek out regulation should surprise nobody familiar with business, politics, or economics. The Obama administration in which Emanuel served was famous for building regulation friendly to—and often at the behest of—various business sectors. The practice was so infamous that Tim Carney (the brother of one of the co-authors of Breitbart Business Digest) even coined the term Obamanomics to describe it.
Dodd-Frank is probably the best example. Instead of bringing the Too Big To Fail banks down to size, it built a fortress of regulation around them and all but guaranteed protection from competition. Was Emanuel never in the room where it happened?
What George Stigler Taught Us in 1971
Regulation as cartelization is nothing new. George Stigler, the University of Chicago economist who won the Nobel Prize in economics in 1982, long ago taught us that businesses frequently enlist government power to improve their market position and stifle competition. His landmark 1971 paper declared that “as a rule, regulation is acquired by the industry and is designed and operated primarily for its benefit.”
It’s important to note that the popular conception of “regulatory capture” actually misses the mark. Under this narrative, the great danger business interests pose to regulation is that because of their influence, a virtuous and independent agency founded to protect the public can fall under corporate influence—becoming captured. This is the sort of mental model that demands limits on corporate “speech” and vigilant protection of agency authority and independence.
Stigler explained that this gets things the wrong way around. Typically, it is an industry’s big incumbent players that demand the creation of the regulatory machinery itself, seeking rules that restrict entry and make competing alternatives more expensive. The corruption is right there from the beginning. Note that this doesn’t have to be a government agency. And it doesn’t have to create a monopoly. It can be a “self-regulatory” body that is given various credentialing privileges and anti-trust exemptions that allow it to cartelize an industry oligopoly.
You don’t have to squint your eyes very hard to see that this looks exactly like what is happening with Amodei and Altman’s call for AI regulation. Amodei wants rules covering frontier companies unwilling to cooperate voluntarily. He also floats capability checkpoints requiring certifications that models possess specified safety properties.
Consider the competitive consequences. A large laboratory with extensive safety staff and centralized operations can accommodate resident evaluators and elaborate certification procedures. A smaller challenger faces a substantial additional cost before it can compete. The rules themselves can be so dense that it requires an army of lawyers to interpret them.
Open-weight AI models, whose underlying parameters can be downloaded and modified, are a particular challenge to the centralized models of OpenAI and Anthropic. Not surprisingly, they’ll likely be targeted by AI licensing and regulation. A regulatory system built around continuing supervision of a company’s centrally controlled product could be especially difficult to apply to software distributed among independent users. That makes it easy to describe open-weight as a “loophole” in the regulatory scheme, creating a rationale for clamping down on their development.
Even “pacing” or “pausing” AI development easily becomes a convenient argument for expanding the regulatory system. A slowdown among a few laboratories leaves outsiders free to advance. That limitation becomes the justification for bringing everyone under rules negotiated by the companies already in front. The labs can argue that no further progress — especially from competitors — is safe unless it is closely monitored. An estimate like Hubinger’s chance of human extinction does that work. It locates the danger in systems not yet built, which is exactly the development the companies already in front have not finished and their rivals have not started.
Note that none of this requires OpenAI or Anthropic to seek exemptions from the regulation they seek for their industry. They can accept every requirement and still benefit when compliance consumes a larger share of a rival’s resources or obstructs a competing business model. Their willingness to pay is perfectly consistent with Stigler’s explanation.
The Wizards of Effective Altruism
Financial commentator Izabella Kaminska has supplied the most vivid description of the current call for regulation. She sees a contest between people who believe the technological magic should be available to everyone, including the muggles, and those who believe its practice should be reserved for the wiz-kids or wizards deemed capable of controlling it. Who gets to wield the technology is the central question.
Kaminska also points out that there is an ideological fight beneath the surface. Many of the top AI executives and developers are devotees of something called effective altruism, or EA, a movement supposedly devoted to identifying how money and effort can do the most good. In reality, however, it is a cold-blooded and arrogant faith in the ability of technocrats to improve the world, a kind of socialism for the cognitive elite. Its moral calculus is often a kind of crude utilitarianism, disregarding traditional concepts of rights and duties in favor of engineering an allegedly better future.
If that sounds more like the ethical code of a supervillain, that’s because it is. It should come as no surprise that Sam Bankman-Fried, the convicted crypto scammer, was a long-time adherent to the philosophy that rules don’t apply so long as you are doing good.
The EA philosophy and its wealthy AI founder-backers has helped supply the AI safety movement with funding, researchers, and an intellectual framework. EA-linked philanthropy has financed work on AI risks for years, helping turn fears about superintelligence into an organized research and policy agenda.
This week, the Pentagon drew attention to the dangers of the EA wizards. “Americanism, not effective altruism,” the Department of War CTO account declared Monday. “The United States will continue to be AI DOMINANT!” This was a not-too-subtle rejection of the calls for an AI pause.
None of this requires the safety warnings to be insincere. Amodi, Altman, and their employees may genuinely fear their technology while favoring a regulatory structure that protects their commercial position. The question is what the proposed rules accomplish, including whom they prevent from competing.
The wizards may have excellent reasons to fear a dangerous spell. That hardly establishes their entitlement to a government-enforced oligopoly and licensing regime on the craft.