The gpt-oss Blossom

One of the most inspiring facets of an open weight model release isn’t that it sparks innovation - it’s that creativity is almost guaranteed. With each open release, we don’t just see usage grow, we see an organic ecosystem blossom - an apt metaphor given the examples below relate to the recent gpt-oss release from OpenAI with their Blossom logo.

AI-generated image (OpenAI DALL-E)

OpenAI gpt-oss

OpenAI released gpt-oss-120b and gpt-oss-20b on 5 August 2025, its first open-weight models since GPT-2, under the Apache 2.0 license and OpenAI’s gpt-oss usage policy.

Both are Mixture of Experts (MoE) models. Instead of activating every parameter for every token, MoE models route each input through a small subset of “experts.”

This makes them far more efficient to train as well as run, and opens new ways to scale models without the linear compute cost of traditional dense LLMs.

These models seem to be sized for cloud inference, with OpenAI stating that the 120b parameter variant runs “efficiently on a single 80 GB GPU”. Even the 20b parameter variant looks too large for most consumer hardware, with OpenAI noting that the 20b variant can run with 16 GB of VRAM.

But predictably, the community responded with ingenuity.

Mixture of Experts (MoE) Offload

Thanks to llama.cpp’s Mixture of Experts (MoE) offload (--cpu-moe flag), the community are running gpt-oss-120b on systems with as little as 8–9 GB of VRAM, streaming expert layers to RAM - albeit requiring 64 GB+ of system RAM.

Whilst actual throughput will vary with CPU speed, RAM bandwidth, storage, and context length, the performance looks to be usable, with references of ~17–25 tokens/sec throughput.

This is unexpected enough that it might even cannibalise a slice of ChatGPT usage at the margins.

Expert Pruning

Aman Priyanshu and Supriti Vijay analysed expert activations in gpt-oss-20b and pruned under-utilised experts across domain-specialised variants, producing ~4.2b to ~20b models spanning 1–32 experts.

This sees variants of gpt-oss that look to be performant but are considerably lighter weight.
Interestingly, quantisation and distillation usually get most of the attention when it comes to compressing models for smaller inference footprints. With this work I imagine pruning will get more attention, especially coupled with Nvidia’s use of pruning in their Minitron work.

Why This Matters

Open weight models provide public access to their trained parameters, allowing users to download, adapt, and fine-tune them for specific tasks without needing the original training data or code.

These models transform flagship releases into fertile ground for community experimentation. The base models are often excellent themselves - but it is the downstream community effort - offloading hacks, pruning pipelines, distillation tangents - that can unlock unexpected uses.

References and Links

OpenAI release

• https://openai.com/index/introducing-gpt-oss/

MoE offloading (--cpu-moe)

• https://github.com/ggml-org/llama.cpp/discussions/15396
• https://www.reddit.com/r/LocalLLaMA/comments/1mke7ef/120b_runs_awesome_on_just_8gb_vram/
  (Note - given this is a Reddit reference, I wouldn’t click through if “direct” language isn’t something you’re comfortable with)

Pruning and expert fingerprinting

• https://github.com/AmanPriyanshu/GPT-OSS-MoE-ExpertFingerprinting
• https://huggingface.co/collections/AmanPriyanshu/gpt-oss-pruned-experts-42b-20b-if-science-math-etc-689c380a366950b1787a20c6
• https://huggingface.co/AmanPriyanshu/collections

Nvidia Minitron

• https://developer.nvidia.com/blog/how-to-prune-and-distill-llama-3-1-8b-to-an-nvidia-llama-3-1-minitron-4b-model/