Regulation Is the Friend of the Incumbent

It's Thursday, July second. So there's a headline in the news right now that caught my attention: "The Anthropic Fable Ban Is Over. The Battle Over How to Tame AI Has Just Begun." And I want to talk about it because it connects to something much bigger than one company or one AI model. Quick context if you missed it: Anthropic, the company that makes Claude, released its most powerful AI models, Fable 5 and Mythos 5, on June 9th. Three days later, the Commerce Department ordered them taken offline, citing national security concerns over a potential jailbreak vulnerability. It was the first time the US government had used export controls to pull a commercial AI product off the market. The ban was lifted this week after two weeks of negotiations. And I should be transparent here: Claude, the AI made by Anthropic, is one of the tools I use to research this show. So the technology behind HigherIQ was briefly banned by the US government. But the reason I want to cover this isn't the ban itself. It's what the ban reveals about a pattern that shows up every time a new technology gets powerful enough to matter. On June 9th, Anthropic launched Fable 5, which it described as its most capable model ever, along with a more powerful research model called Mythos 5. Anthropic had spent thousands of hours red-teaming the models with the US and UK governments, private third-party organizations, and internal teams before release. They built guardrails to prevent the models from assisting with cybersecurity attacks and biological threats. They implemented a 30-day data retention policy so they could monitor for misuse. By their account, no testers had found a universal jailbreak, meaning a method that could broadly bypass the model's safety systems.

Three days after launch, on June 12th, the Commerce Department sent Anthropic a letter at 5:21 PM Eastern time directing the company to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States. The letter cited national security authorities but didn't provide specific details about the concern. Anthropic's understanding was that government officials believed they'd found a jailbreak, a way to bypass the model's safety guardrails. Anthropic reviewed the technique and concluded it was a narrow vulnerability that other publicly available AI models could also exploit, not something unique to Fable 5.

Anthropic complied with the order but publicly disagreed with it, writing: "If this standard was applied across the industry, we believe it would essentially halt all new model deployments for all frontier model providers." They sent a team of their top scientists to Washington to work with government officials on a resolution. Two weeks later, Commerce Secretary Howard Lutnick sent a letter lifting the export controls, citing Anthropic's cooperation. Fable 5 came back online yesterday.

The immediate story is straightforward: government saw a risk, pulled a product, company negotiated, product came back. But there's a deeper pattern at work here, and a venture capitalist named Bill Gurley explained it better than anyone before any of this happened. In September 2023, Bill Gurley, a general partner at Benchmark who had invested in Uber, Grubhub, and Zillow, gave a presentation at the All-In Summit titled "2,851 Miles," which is the distance between Silicon Valley and Washington DC. He had the audience chant a sentence that doesn't normally generate excitement: "Regulation is the friend of the incumbent."

Gurley's argument, which builds on the work of George Stigler, who won the Nobel Prize in Economics in 1982, is that regulation almost always ends up benefiting the largest companies in any industry, regardless of what the regulation was designed to do. Stigler's original insight was that "as a rule, regulation is acquired by the industry and is designed and operated primarily for the benefit of the industry." Not the public. The industry. The companies being regulated are the ones who shape the rules, because they have the lobbyists, the relationships, and the resources to influence the process. Startups don't have lobbyists. They barely have lawyers.

Gurley walked through example after example. When the federal government pushed hospitals to adopt electronic health records under Obama, it effectively mandated the feature set of Epic Systems, the dominant player. Epic's software became the compliance standard. Three of Epic's smaller competitors were hit with fines between $57 million and $155 million. The regulation that was supposed to modernize healthcare locked in the incumbent and crushed the upstarts. When Verizon and Comcast saw municipal WiFi projects threatening their business, they lobbied state legislatures to outlaw them within two years. When COVID rapid antigen tests, a simple technology that cost less than a dollar in Europe, came to the US market, regulatory barriers kept the price at ten dollars or more and limited competition to a handful of approved manufacturers.

Then Gurley turned to AI. He showed a New York Times headline: "OpenAI's Sam Altman Urges AI Regulation in Senate Hearing." Gurley's point was that the CEO of the biggest AI company in the world going to Congress and asking for regulation should alarm everyone, because the company most likely to benefit from compliance requirements, licensing standards, and government review processes is the company that already has the scale, the lawyers, and the government relationships to navigate them. A startup building an AI model in a garage cannot afford a team of scientists to fly to Washington for two weeks of negotiations. OpenAI and Google can. If the regulatory bar gets high enough, the only companies that can clear it are the ones that are already on top. The intellectually honest version of this story requires holding two things in your head at the same time, and I think most coverage picks one and ignores the other.

The first thing is that the safety concerns are real. AI models are getting more capable every few months, and the gap between what they can do and what their guardrails prevent is a legitimate area of concern. Anthropic itself has been one of the most vocal companies about AI safety risks. They built an entire research program around it. The government having some mechanism to intervene when a technology poses real national security risks isn't inherently unreasonable. Nobody argues that the FDA shouldn't exist or that nuclear materials shouldn't be regulated.

The second thing is that the mechanism matters enormously, and the mechanism used here was export controls, the same legal tool the government uses for weapons, military technology, and nuclear equipment. Applied to a chatbot. With three days' notice. With no specific technical details provided in the initial letter. And with a standard that, as Anthropic pointed out, would shut down every frontier AI model if applied consistently. That's not a transparent regulatory process. That's a phone call that can kill a product overnight, and the companies best positioned to survive that phone call are the ones with the deepest government relationships and the biggest compliance budgets.

The Anthropic situation is interesting precisely because Anthropic isn't a scrappy startup. They're well-funded, well-connected, and they were able to send scientists to DC and negotiate a resolution in two weeks. The question Gurley would ask is: what happens when this tool gets used against a company that can't do that? What happens when a smaller AI lab releases a model that competes with the incumbents, and a jailbreak vulnerability gets flagged, and the Commerce Department sends a letter at 5:21 PM on a Thursday? Does that company have the resources to fly a team to Washington and negotiate for two weeks while their product is offline and their customers leave? Or does the product just stay dead?

That's the pattern Gurley described. The regulation doesn't need to be designed to help the incumbent. It just needs to cost enough to comply with that only the incumbent can afford it. The safety concern can be completely valid, and the competitive effect can still favor the biggest players. Both things are true at the same time, and the fact that they're hard to separate is exactly what makes regulatory capture so effective and so difficult to fix. So if this comes up in conversation, here's how to think about it. The US government used export controls to pull Anthropic's most powerful AI model offline for two weeks, the first time that tool has been used against a commercial AI product. The ban was lifted after negotiations. Bill Gurley warned about this dynamic years ago at the All-In Summit: regulation almost always benefits the largest players in an industry, because they're the ones who can afford to comply. The safety concerns with AI are real. The mechanism used to address them, export controls with no transparency, three days' notice, and a standard that would shut down every frontier model if applied consistently, raises questions about who that mechanism actually protects. The hard part is that both things can be true simultaneously: the technology can be a real risk and the regulatory response can still favor the incumbents. Telling the difference between safety and competitive advantage is the challenge every new technology eventually faces. AI just got there faster than most.

Stay informed, stay curious, and we'll see you tomorrow.