Research reproduction audit
A de-identified, call-by-call projection of a captured repository audit followed by its machine grade.
Audit a one-dimensional convection PINN reproduction packet and classify whether its reported evidence supports the central claim.
Captured run, de-identified01 / 05Task
Toolbash
- Intent
Inspect the benchmark packet
Tool call
Excerpt from the recording
List the workspace and inspect the benchmark PDE module; source contents are withheld.
ls -la ls submission/pinn/
Observed resultok
Summary
The benchmark package, submission directory, and task file are present; the task defines a one-dimensional convection PINN audit.
- Intent
Check the packet format
Tool call
Excerpt from the recording
List submission files and look for result and JSON rubric files.
ls submission/ find . -name "result.json" 2>/dev/null find . -name "*.json" | head
Observed resultok
Excerpt from the recording
The submission files and rubric JSON are present; no precomputed result is used.
pinn reproduce.sh requirements.txt train_pinn.py
- Intent
Write the auditor
Tool call
Withheld: generated auditor source
Write the normalized auditor artifact.
Observed resultok
Summary
The requested auditor artifact was written; its original output filename is withheld behind a stable public alias.
- Intent
Exercise failure paths
Tool call
Withheld: generated test code
Run the local failure-path test harness.
Observed resultok
Excerpt from the recording
The recorded local checks classify each path according to the fixed priority order.
Recorded output, 7 lines
valid: valid-reproduction fab: fabricated-metrics bnd: boundary-condition-mismatch res: residual-not-satisfied nocurv: no-curvature-gain insuff: insufficient-final-accuracy 0.1419 noreport: fabricated-metrics
- Intent
Submit the audit run
Tool call
Withheld: final agent summary
Submit the completed run.
Observed resultsubmitted
Excerpt from the recording
The run finished and handed off to the machine grader.
done
Pass
The captured auditor passes every held-out reproduction-audit check.
- Score
- 1.0
- Threshold
- 1.0
- Held-out checks
- 6 / 6
- Held-out check 1Pass
- Held-out check 2Pass
- Held-out check 3Pass
- Held-out check 4Pass
- Held-out check 5Pass
- Held-out check 6Pass