MMLU (5-shot)Knowledge
84.2/ 100
Verified
Last Verified: Unknown DateGoogle AI Blog
Massive Multitask Language Understanding covers 57 subjects across STEM, the humanities, social sciences, and more.
Beta version: *Information might not be fully accurate. Please report any discrepancies.
Latest Data
Unknown
Context Window
1.0M
tokens
Input Cost
$0.10
per 1M tokens
Output Cost
$0.40
per 1M tokens
Cache Cost
$0.03 / Free
read / write per 1M
Parameters
Multimodal Live
model footprint
Performance Analysis // Verified Benchmarks
Massive Multitask Language Understanding covers 57 subjects across STEM, the humanities, social sciences, and more.
Resolving real-world GitHub issues. Verified subset ensures solvable issues.
Comprehensive framework to evaluate LLMs as agents across diverse environments.