Independent benchmark
Model Capability
Coding capability is measured from executable repository tasks and their test results. Cost, latency, tokens, and cache behavior stay separate from the capability score.
Awaiting the first benchmark run
No sample values are published. The page will activate after an external runner completes at least ten fixed coding tasks and uploads their test outcomes.
Capability score
Passed tasks receive difficulty weights derived from each task’s historical pass rate. A harder passed task contributes more.
Degradation verdict
Each run is compared with the median of the previous fourteen publishable runs using the same model, effort, and task suite.