PreviewFull dataset construction in progress

NLL Physics Research 1

Physics Research Grade problems are research-level physics and mathematics derivations, each reconstructed from a recent arXiv preprint and scored only on whether a model reaches the exact terminal result — a direct read on reasoning ability rather than style.

Pass@1

Strict — whole problem solved

Rubric score

Weighted partial credit

Each problem is broken into weighted subquestions, and the rubric score is the fraction of that credit an answer earns. It gives a model credit for correct structure and intermediate results even when it misses the final answer — where pass@1 above demands the complete result, this is the softer, partial view.

Cost vs performance

Avg $ per task · pass@1

Each mark is one model, placed by the selected x-metric (log scale) against its strict pass@1 — cheaper or leaner to the left, stronger upward, so the top-left is the efficient frontier. Hover a mark for its full figures. Qwen's cost went untracked on its free route, so it is omitted from the cost view but still appears under the token axes.