MMLU (5-shot)Knowledge
86.5/ 100
Verified
Last Verified: 2025-09-30Z.ai Blog
Massive Multitask Language Understanding covers 57 subjects across STEM, the humanities, social sciences, and more.
Beta version: *Information might not be fully accurate. Please report any discrepancies.
Latest Data
2026-02-20
Context Window
200k
tokens
Input Cost
$0.10
per 1M tokens
Output Cost
$0.30
per 1M tokens
Parameters
355B MoE
model footprint
Performance Analysis // Verified Benchmarks
Massive Multitask Language Understanding covers 57 subjects across STEM, the humanities, social sciences, and more.
Challenging competition mathematics problems (AIME/IMO level).
Functional correctness of synthesized programs from docstrings.
Resolving real-world GitHub issues. Verified subset ensures solvable issues.
Contamination-free, continuously updated reasoning benchmark.
Chatbot Arena ELO score. Crowd-sourced human preference ranking.
Graduate-Level Google-Proof Q&A Benchmark.