ByteDance releases new open source Seed-OSS-36B model

In releasing both synthetic and non-synthetic versions of the Seed-OSS-36B-Base model, the Seed Team sought to balance practical performance with research flexibility.

The synthetic-data variant, trained with additional instruction data, consistently delivers stronger scores on standard benchmarks and is intended as a higher-performing general-purpose option.

The non-synthetic model, by contrast, omits these augmentations, creating a cleaner foundation that avoids potential bias or distortion introduced by synthetic instruction data.

By providing both, the team gives applied users access to improved results while ensuring researchers retain a neutral baseline for studying post-training methods.

Meanwhile, the Seed-OSS-36B-Instruct model differs in that it is post-trained with instruction data to prioritize task execution and instruction following, rather than serving purely as a foundation model.

All three models are released under the Apache-2.0 license, allowing free use, modification, and redistribution by researchers and developers working for enterprises.

That means they can be used to power commercial applications, internal to a company or external/customer-facing, without paying ByteDance any licensing fees or for application programming interface (API) usage.

This continues the summer 2025 trend of Chinese companies shipping powerful open source models with OpenAI attempting to catch up with its own open source gpt-oss duet released earlier this month.

The Seed Team positions Seed-OSS for international applications, emphasizing versatility across reasoning, agent-like task execution, and multilingual settings.

The Seed Team, formed in 2023, has concentrated on building foundation models that can serve both research and applied use cases.

Design and core features

The architecture behind Seed-OSS-36B combines familiar design choices such as causal language modeling, grouped query attention, SwiGLU activation, RMSNorm, and RoPE positional encoding.

Each model carries 36 billion parameters across 64 layers and supports a vocabulary of 155,000 tokens.

One of the defining features is its native long-context capability, with a maximum length of 512,000 tokens, designed to process extended documents and reasoning chains without performance loss.

That’s twice the length of OpenAI’s new GPT-5 model family and is roughly equivalent to about 1,600 pages of text, the length of a Christian Bible.

Another distinguishing element is the introduction of a thinking budget, which lets developers specify how much reasoning the model should perform before delivering an answer.

It’s something we’ve seen from other recent open source models as well, including Nvidia’s new Nemotron-Nano-9B-v2, also available on Hugging Face.

In practice, this means teams can tune performance depending on the complexity of the task and the efficiency requirements of deployment.

Budgets are recommended in multiples of 512 tokens, with 0 providing a direct response mode/

Competitive performance on third-party benchmarks

Benchmarks published with the release position Seed-OSS-36B among the stronger large open-source models. The Instruct variant, in particular, posts state-of-the-art results in multiple areas.

• Math and reasoning: Seed-OSS-36B-Instruct achieves 91.7 percent on AIME24 and 65 on BeyondAIME, both representing open-source “state-of-the-art” (SOTA).
• Coding: On LiveCodeBench v6, the Instruct model records 67.4, another SOTA score.
• Long-context handling: On RULER at 128K context length, it reaches 94.6, marking the highest open-source result reported.
• Base model performance: The synthetic-data Base variant delivers 65.1 on MMLU-Pro and 81.7 on MATH, both state-of-the-art results in their categories.

The no-synthetic Base version, while slightly behind on many measures, proves competitive in its own right.

It outperforms its synthetic counterpart on GPQA-D, providing researchers with a cleaner, instruction-free baseline for experimentation.

For enterprises comparing open options, these results suggest Seed-OSS offers strong potential across math-heavy, coding, and long-context workloads while still providing flexibility for research use cases.

Access and deployment

Beyond performance, the Seed Team highlights accessibility for developers and practitioners. The models can be deployed using Hugging Face Transformers, with quantization support in both 4-bit and 8-bit formats to reduce memory requirements.

They also integrate with vLLM for scalable serving, including configuration examples and API server instructions.

To lower barriers further, the team includes scripts for inference, prompt customization, and tool integration.

For technical leaders managing small teams or working under budget constraints, these provisions are positioned to make experimentation with 36-billion-parameter models more approachable.

Licensing and considerations for enterprise decision-makers

With the models offered under Apache-2.0, organizations can adopt them without restrictive licensing terms, an important factor for teams balancing legal and operational concerns.

For decision makers evaluating the open-source landscape, the release brings three takeaways:

• State-of-the-art benchmarks across math, coding, and long-context reasoning.
• A balance between higher-performing synthetic-trained models and clean research baselines.
• Accessibility features that lower operational overhead for lean engineering teams.

By placing strong performance and flexible deployment under an open license, ByteDance’s Seed Team has added new options for enterprises, researchers, and developers alike.

Daily insights on business use cases with VB Daily

If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.

Read our Privacy Policy

Thanks for subscribing. Check out more VB newsletters here.

An error occured.