There's a category error running through the current market for agent data. Benchmarks are being sold as reinforcement-learning environments, and they aren't the same object.
A benchmark answers a question about a finished attempt: did it work. An environment has to answer a question about a partial attempt: is this better than the last one. Those requirements diverge, and a task built for the first is usually useless for the second.
Why pass/fail is a bad reward
Consider a long-horizon task where a competent attempt gets most of the way and then fails on one behavior. Under pass/fail grading that attempt scores zero, identical to an attempt that did nothing.
The gradient is gone. Across a training run this produces the familiar pathology: on hard tasks almost everything scores zero, the reward carries no information about which direction was better, and the policy learns from the rare accidental success rather than from the systematic near-miss. The longer the horizon, the worse it gets, because the probability of a complete success falls off while the informativeness of partial progress rises.
Which is precisely backwards for the tasks anyone actually cares about.
Assertion fractions, subtask ladders, substep trees
Dense reward has to be built into the task. It can't be added by the training harness afterward, because the harness has no way to know what partial progress means. Three constructions cover most cases.
Assertion fractions. Where the task has a held-out test suite, score the fraction of assertions passed rather than the conjunction. An honest eighty-percent implementation scores 0.80. Keep a separate ship bar at full pass for when you need a binary answer, so the same task serves as both an environment and an evaluation without being rebuilt.
This requires knowing the exact assertion count, which means counting it from the generated suite rather than estimating it. In our repository lane those counts range from thirty to nearly two hundred per task, and the count is written into the grading contract as the pass threshold. An approximate denominator produces an approximate reward.
Subtask ladders. Where the task is a chain that has to be walked in order, ship per-stage checks. A security target that requires injection, then privilege escalation, then data export, should report that a run reached stage two. Under pass/fail it reports nothing, and stage two is exactly the observation that tells you the run was on the right path.
Substep trees with skip propagation. Where the task is building something whole, decompose it into a tree of user-visible behaviors, each independently checkable. Nodes whose prerequisites are missing get marked skipped rather than failed, which distinguishes "attempted and wrong" from "never reached". Those are different signals and collapsing them into a single failure destroys most of what the run told you.
A broken grader still returns plausible numbers
Dense reward introduces a failure mode that binary grading doesn't have. A partially-credited grader can be quietly broken in a way that still produces plausible numbers, and you will read those numbers as model behavior.
Two checks catch it, and they should run before the environment ships.
A correct reference implementation must score all-green. If it doesn't, the ceiling isn't where you think it is, and every score below it is compressed against a wall you didn't know about.
A deliberately incomplete stub must score partial, with skips propagating correctly. If the stub scores zero, your partial credit is not working and you have a pass/fail grader wearing a fractional score. If it scores near-full, your checks are too weak to discriminate and the reward is noise.
Run both on every task. When a scoring distribution shifts afterward, you will know it reflects the model rather than the harness, which is the entire reason to bother.
What exactly is correct
Partial credit is easy to describe and hard to make deterministic, and the difficulty is always the same: what exactly is correct.
Where the ground truth is authored, this is tractable. If you inject a known fault into real operating data, you know both the exact end value and the exact record responsible, so you can score the arithmetic and the diagnosis separately. That separation matters more than it sounds. A run that produces the right number by a wrong route is not a run that understood the task, and scoring them apart is what stops a lucky answer from reading as a diagnosis.
Where the ground truth is a judgement, you need a rubric with enough leaves to be dense, and then you need to measure the judge against the rubric rather than trusting it. A judge that's merely asserted to be reliable is a reward function nobody has evaluated, sitting at the center of a training run.
Three questions that separate the two
For any task set being sold as an environment rather than a benchmark, three questions separate the two.
What does a half-correct attempt score, and how is that number computed. What proves the grader discriminates, meaning what does a known-good and a known-bad submission each score. And is the reward deterministic, or does it depend on a model whose agreement with the intended standard hasn't been measured.
A vendor who can't answer these is selling you a measurement. That may be exactly what you want, and it's worth knowing which one you're buying.