Open-Weight Models Aren’t Enough. We Need Truly Open Source AI Models for Science and Society.

Chinese open-weight models have made stunning progress in recent months, with offerings from companies like DeepSeek and Alibaba closing the performance gap with American counterparts. Stanford HAI’s own AI Index found the U.S.-China performance gap narrowing to a few percentage points last year. Two new Chinese models narrow this gap even more. Moonshot AI’s Kimi K3, a 2.8-trillion-parameter model with a million-token context window featuring 2.4 trillion parameters, and Alibaba’s Qwen3.8-Max, featuring 2.4 trillion parameters, are the most powerful open-weight models yet, edging closer in the rankings to the highest-performing American models. These new models demonstrate big improvements in open-weight AI performance while costing less than top-tier American AI models. 

Meanwhile, U.S. policymakers are increasingly focused on the cybersecurity and national security risks posed by frontier AI. The current U.S. administration has shifted from an “innovation first” position to a more reactive one – requiring advance notice of new model releases, and, in one case, forcing Anthropic to take Mythos 5 and Fable 5 fully offline under a Commerce Department export-control order. With Chinese labs now closing the capability gap, the administration is weighing whether to extend that kind of scrutiny to Chinese open models as well.

The conversation has led to an American corporate response; Nvidia, Microsoft, Meta, and 20-plus other companies published an open letter warning Washington against “premature restrictions” on open-weight AI models, highlighting the need for competitiveness in a heated global market.

Stanford Institute for Human-Centered AI Denning Director James Landay agrees with the instinct behind that letter. He just thinks it stops short of the point.

“Open weights are progress,” he said. “You can download the model, run it on your own machine, keep it out of someone else’s data pipeline. But you still can’t see how the thing was built, what it was trained on, or why it behaves the way it does. That’s not an open model. That’s open distribution.”

The Value of Truly Open Source Models for Science and Society

Defining open source AI is complicated. Free and open source software has a settled vocabulary – a license either grants you the rights or it doesn’t. AI has no equivalent consensus, because the artifact itself is broader than a codebase: It’s software, training and test data, and learned parameters bundled together, and each piece can be open or closed independently. A company can release the weights, withhold the data and training code, and still call the result “open.”

Calling yourself “open” carries connotations of trust, Landay says. But companies would need to hand over the training data before outsiders could actually test, reproduce, or challenge the work. “There’s a wide gap between open-weight AI and open source AI.”

He says Stanford HAI has staked out a specific bar: alignment with the Linux Foundation’s Model Openness Framework at its top tier, the “Open Science” class – meaning the code, the training data (or a thoroughly documented, auditable account of it), and the tooling are all released, not just the weights, alongside a real way for outside researchers and communities to download, run, study, contribute to, and modify the work. Anything short of that, in his view, forecloses entire lines of research: You can’t, for example, study how a model’s training data shapes its cultural assumptions, audit it for how fairly it represents different populations, or test new architectures at the scale where genuinely new behavior shows up. You can only observe a finished system from the outside and guess at why it does what it does.

Landay adds that nearly every major architectural leap in modern AI – the transformer itself, attention mechanisms, and mixture-of-experts routing – came out of published research that other labs and academics could pick apart, question, and build on. Radical innovation depends on open science. Once frontier work moves behind closed doors, the next generation of researchers can only iterate on what a handful of companies choose to disclose.

“Open weights answer ‘Can I run this?’” Landay said. “Open source answers ‘Can I trust this, improve it, and build the next thing on top of it?’ Right now almost everyone – American labs and Chinese labs alike – is answering the first question but nowhere close to the second.”

The Risk of Closed AI 

Closed models – whether American or otherwise – are not necessarily less safe in the moment, but carry risks of concentration. A small number of firms controlling frontier capability behind an API means a small number of firms deciding who gets access, at what price, and whose values get baked into a tool that increasingly mediates how people work and think.

He traces this to a familiar pattern: platforms that start out creating value for users and gradually pivot toward extracting it – what Cory Doctorow calls “enshittification.”

“A closed frontier model behind an API is the ultimate walled garden,” Landay said. “You don’t own your data, your history with the model, or any leverage to leave. That’s already the norm.” The longer someone stays inside one of those gardens, he added, the more of their own context and history the platform accumulates – which only raises the cost of ever leaving, deepening the lock-in rather than easing it over time.

There’s a security argument here too: Relying solely on closed models isn’t inherently safer, since they can be breached, misused, or fail in ways outsiders have no way to detect. Concentrating frontier capability behind a handful of closed systems just means those blind spots are concentrated too.

Closed models also risk encoding a narrower slice of the world’s values than the world they’re meant to serve. A handful of labs – whether in the U.S. or China – training frontier systems mostly on their own data, in their own language, under their own assumptions, produce tools that quietly carry one culture’s worldview into everyone else’s daily life. 

And because frontier capability sits with so few companies concentrated in so few countries, it becomes a strategic chokepoint as well. Landay points to Anthropic’s own latest models going dark under a Commerce Department order as a preview of how quickly access to closed frontier AI can become a bargaining chip in a trade dispute or export-control fight, with the people relying on it caught in the middle.

Defining an Open Approach

Landay’s fix has three parts: 

• Decentralized AI: AI you can run locally or on your own servers rather than rent

• Data portability: Your context and data travel with you between platforms instead of locking you in

• Verifiable agents: AI whose builders, loyalties, and actions are actually verifiable.

None of that works, though, if the underlying models are still built behind closed doors by a handful of firms. Landay thinks closing that gap is a job for a different kind of institution altogether: universities. Academic institutions can commit to timelines longer than a product cycle, publish work meant to be checked rather than merely used, and pursue questions with no clear path to revenue – building and studying leading-edge systems themselves, not proxy models or API access to someone else’s black box.

“If the future is genuinely open, in the full sense, it doesn’t matter as much whose flag is on the release, because anyone anywhere can inspect it, adapt it, and hold it accountable,” he said. “If ‘open’ just means ‘downloadable,’ we’ve traded one set of closed labs for another. Same concentration of power, different flag. Tomorrow’s AI won’t just complete tasks for us – it’ll shape what we consider possible. The question was never really ‘Can we build this?’ It’s ‘What world are we building, and whose humanity are we encoding into it?’ Open weights alone won’t answer that. Only open source AI for science and society will.”