
That matters because open-weight models give people the freedom not to rely on a company’s subscription services. They can be deployed on infrastructure controlled by the customer, modified for specialised tasks, and operated at lower costs. But to the discomfort of U.S. companies, many of them are based in China. And it’s at that point that AI safety gets muddled with geopolitics.
In July, Axios reported that OpenAI and Anthropic, despite their fierce commercial rivalry, had found a common cause: warning U.S. policymakers about the dangers of powerful Chinese open-weight models. Kristy Loke, an AI governance researcher at King's College London who specialises in China's approach to the technology, questions the timing. "It's very suspicious timing asking for pacing," she told Tom's Hardware Premium , referring to calls for slowing frontier AI development just as Chinese open-weight models are undercutting their American rivals on price.
Loke believes companies genuinely worry about the risks. But she also thinks they're worried about their commercial prospects. "they're calling for pacing because they have some safety concerns, they have some commercial anxiety," she said.
She believes some influential people in American AI have increasingly sought to "paint open models in a very bad light", portraying them as inherently less secure than closed alternatives. Yet the risks of AI systems misbehaving aren't restricted to models developed in China, as OpenAI's own incidents demonstrate.
But open-weight models come with their own risks. Because their underlying parameters can be modified , third parties can create and distribute "abliterated" models, removing the refusal behaviours designed to prevent AI from responding to potentially harmful requests. Many of these models are available on platforms including Hugging Face. “They’re not adding in new behaviour; they’re just removing this refusal behaviour,” said Catherine Flick, professor of ethics and games technology at the University of Staffordshire, in an interview with Tom’s Hardware Premium . Abliterating models doesn’t make them more capable, Flick said, but it does mean they can answer questions they previously wouldn’t have been willing to. That also means those responses might not have been tested for accuracy or hallucinations.
Those in favour of open-weight models like DeepSeek and Qwen say that removing those guardrails helps security researchers understand how AI behaves. Flick said that’s plausible, but “I just don’t really buy it,” she said. She reckons that the negative potential outputs outweigh the positive case for abliteration because when they’re publicly available, preventing their misuse becomes tricky. American labs clearly have commercial reasons to criticise open-weight competitors because the latter undercuts their economics, but it also doesn’t mean their concerns about safety aren’t valid, too.
Von Thun fears that could result in precisely the wrong kind of regulation. "I think what we should not be doing is allowing the companies to basically form a safety cartel," he said.
Yet U.S. AI companies benefit from the government's determination to maintain its advantage over China. Von Thun argues the Trump administration has created an environment in which economic power translates into political influence, allowing major technology companies to invoke national security concerns when arguing for policies that may also serve their commercial interests. But that could backfire. "I think completely closing off the market to AI products from other countries, including China, would not be the right approach," von Thun said.
Part of the reason that wouldn’t be the right approach is that the idea that the United States is focused on developing AI responsibly while China isn’t is a fallacy. Just earlier this month, Reuters reported that ByteDance had established a dedicated Alignment & Preparedness team, recruiting researchers to investigate safety risks arising from increasingly capable AI systems.
For von Thun, the solution is to prevent either industry or geopolitical competition from determining what safety means. He points to the European Union's AI Act, which establishes obligations for developers of general-purpose AI models classified as posing systemic risks. The obligations require companies to conduct model evaluations, adversarial testing, cybersecurity measures and also to report serious incidents.
The opposition to the EU’s AI Act from American tech companies, U.S. politicians and the general public who grouse about the way it delays access to cutting-edge models for Europe’s 450 million citizens demonstrates that these provisions don't automatically solve the alignment problem. But they at least involve more people in the conversation about what is and isn’t acceptable for AI than just the industry's biggest participants.
Even when AI companies agree to independent oversight, they still want some say over how it works — and their own internal decision-making on what is and isn’t safe differs from that of those outside the company. Loke warns that leading AI companies are experimenting with recursive self-improvement, where models help develop their successors, potentially making it more difficult for humans to understand how decisions are being made. "They’re still attempting more risky methods to train AI that will make the AI harder to control," she said.
Von Thun also believes competition rules and interoperability requirements could help prevent AI companies from locking customers into proprietary ecosystems as their agents become increasingly embedded in everyday workflows.
There are plenty of good reasons to be concerned about alignment, particularly when the models being developed are becoming more capable and more difficult to monitor. But letting the companies responsible for building them determine what constitutes an acceptable risk, and who can compete, brings its own dangers. “I’m not saying we don’t need to worry about safety,” said von Thun, “but we should about letting the companies do it."
Chris Stokel-Walker Freelance Contributor Chris Stokel-Walker is a Tom's Hardware contributor who focuses on the tech sector and its impact on our daily lives— online and offline. He is the author of How AI Ate the World, published in 2024, as well as TikTok Boom, YouTubers, and The History of the Internet in Byte-Sized Chunks.
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/tech-industry/artificial-intelligence/SPONSORED_LINK_URL
- https://www.tomshardware.com/tech-industry/artificial-intelligence/ai-safety-legislation-has-a-conflict-of-interest-problem-big-us-labs-are-warning-of-risk-just-as-cheaper-chinese-open-weight-models-erode-their-commercial-edge#main
- https://www.tomshardware.com/my-account
- Leaked Ohio bill could allow AI data center companies to seize private land for their buildouts using eminent domain law
- Samsung, SK hynix, and Micron form unlikely alliance to standardize robot charging as chipmakers push toward unmanned fabs
- Elon Musk’s SpaceXAI suffers chemical spill at Mississippi AI data center
- Minisforum Elite Mini M2 Air-304 review: A quiet and compact Wildcat
- Enhance your PC setup with these great gift ideas under $25
Informational only. No financial advice. Do your own research.