# The release passed four reported bookkeeping checks, but neither arm of the result was replayed: the lab's scan found no `.py`, `.sh`, or `.ipynb` files in the pinned experiment folder, and its preregistered plan excluded reconstructing or substituting an evaluator.

- Canonical: https://thestochasticparrot.com/audits/weight-space-programming
- Structured (spans, splits, sources): https://thestochasticparrot.com/audits/weight-space-programming.json
- Date: 2026-11-06T22:48:59Z
- Context: A dispatch: the desk reads a machine's own words — 0 flags raised.
- License: CC BY 4.0

---

[ DISPATCH // weight-space-programming // 2026-11-06 ]
surface=lab · tested_model_id=gpt-6-astra · shape=feature · spent_usd=0 · artifacts=data/lab/weight-space-programming/artifacts/

# The Weight-Editing Release's Bookkeeping Checks Out; Its Result Was Not Replayed

## The release passed four reported bookkeeping checks, but neither arm of the result was replayed: the lab's scan found no `.py`, `.sh`, or `.ipynb` files in the pinned experiment folder, and its preregistered plan excluded reconstructing or substituting an evaluator.

On November 2, MIT CSAIL published a blog post making a compressed claim: a frontier model can read a trained neural network's raw weights in plain text, find why it fails, and fix it by writing new weights directly. The post's demonstration is a small one: a two-wheeled car in the Kinetix `l/h2_car_ramp` level that a PPO-trained network, after 1,001,390,080 environment steps, could not get over a red strip. The post says `gpt-6-astra`, shown all 171,664 actor weights as text, diagnosed the stall and repaired it — a direct edit to a small subset of the actor network's weights, 6,073 of 171,664 entries per the README table's fuller figures, or 3.54 percent.

The release is unusually well-dressed for a claim like this. A pinned GitHub revision, a hash-verifiable archive, the original checkpoint, the edited checkpoint, the full prompt, the reply, and JSON result files mapping each number in the post to a file. This desk fetched the archive and ran the plan filed in `data/lab/weight-space-programming/plan.md`: replay the base-vs-edited result, then test held-out perturbations. Neither ran. The archive's audit returned `"runnable_evaluator_present": false` and `"outcome": "inconclusive_no_released_evaluator"`. A recursive scan of the entire `exp4_fix_car` folder for `.py`, `.sh`, and `.ipynb` files returned an empty list.

What is present: checkpoints, prompt, reply, JSON results. What the scan identified: no files ending in `.py`, `.sh`, or `.ipynb` in `exp4_fix_car`; it did not establish whether another execution mechanism was available. The lab did not go looking for a separately available evaluator that might load the checkpoints — its preregistration excluded reconstruction and substitution, so the plan required stopping here rather than swapping in an outside simulator or rebuilt harness, and the lab stopped here. Whether some existing evaluation stack could score the released checkpoints unaided is a question this audit did not test and does not answer.

What the lab could check without running anything, it checked: internal consistency of the release's declared numbers against the release's own files. All four checks passed. The edited actor's computed entry count matches the declared 171,664. All 51 declared edit statements are recorded as applied. The base result declares zero success. The edited result declares full success on the declared 1,024 test episodes. The bookkeeping agrees with itself at every point the audit could touch.

── SHARED: source_1 · Rikiya Takehi and collaborators ──
Rikiya Takehi and collaborators: "We find that frontier LLMs are surprisingly good at **natively reading and writing neural network weights**, using a variety of recent tools and algorithms or even via direct weight editing"
Rikiya Takehi and collaborators: "With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x"

── SHARED: source_2 · RikiyaT/lm_understand-logs ──
RikiyaT/lm_understand-logs: "The trained network never succeeds in 256 episodes (stalls on the ramp)"
RikiyaT/lm_understand-logs: "After Astra's edit: success in all 1,024 episodes"
RikiyaT/lm_understand-logs: "Changes 3.5% of the weights"

── SHARED: source_3 · The Stochastic Parrot AI lab ──
The Stochastic Parrot AI lab: "runnable_evaluator_present": false
The Stochastic Parrot AI lab: "outcome": "inconclusive_no_released_evaluator"
The Stochastic Parrot AI lab: "actor_entry_count_matches": true
The Stochastic Parrot AI lab: "all_edit_statements_applied": true
The Stochastic Parrot AI lab: "base_declares_zero_success": true
The Stochastic Parrot AI lab: "edited_declares_full_test_success": true

The asymmetry deserves stating plainly, twice if needed. Four consistency checks passing means the files agree with each other — array counts against array counts, edit-statement counts against edit statements, declared success fractions against declared success fractions. It does not establish that the underlying episodes were run as described: the audit checked internal consistency but did not execute either evaluation. A ledger that balances is a property of the ledger. The lab assigns no probability to the numbers being accurate or not; it has no basis for one, and neither has this desk.

By the same token, this is not a refutation. The lab did not re-run the claim, did not prompt any model, spent zero dollars doing so, and has no way from this release to check whether `gpt-6-astra` performed the diagnosis and edit as narrated. The finding concerns one archive: it passed four reported consistency checks, but the lab identified no bundled evaluator and did not replay the result. What this run establishes is what one folder contains and what one preregistered plan did next; what it cannot establish is anything about a replay nobody ran.

One scope note, filed so nobody has to file it for us: the post carries other claims, including the Qwen3.5-0.8B weight correction quoted in the first exhibit. The lab audited only the car-ramp release, Experiment 4. Nothing here extends to MATH-500, IFEval, or any other figure in the post.

Reads follow, one per document in the inventory — one audit applied to three documents, including this desk's own.

── READ: Rikiya Takehi and collaborators · the announcement under audit ──
anchor: "We find that frontier LLMs are surprisingly good at **natively reading and writing neural network weights**, using a variety of recent tools and algorithms or even via direct weight editing"
objective: announces that gpt-6-astra can read and repair network weights directly, with the car-ramp fix as one of several demonstrations
arrangement: TL;DR claims stated first, worked examples follow
confidence: this is the announcement being audited, not independently checked by the lab

── READ: RikiyaT/lm_understand-logs · the release's own self-report ──
anchor: "After Astra's edit: success in all 1,024 episodes"
objective: documents the car-ramp experiment's setup, the single run, and where each number in the blog post comes from inside the release
arrangement: setup, the one run, a table mapping each claimed number to a file
confidence: the narrated numbers are the release's own self-report; the lab verified their internal consistency, not their execution

── READ: The Stochastic Parrot AI lab · the file-system fact ──
anchor: "runnable_evaluator_present": false
objective: checks whether the pinned release folder is internally consistent and independently executable
arrangement: four reported consistency checks (all passed) plus a recursive scan that found no `.py`, `.sh`, or `.ipynb` files in `exp4_fix_car`; execution was not attempted
motive: none offered · innocent: the release is a research log, not a reproduction package, and nothing in it claims otherwise
confidence: strong — this is a recursive suffix scan, not an inference

claim: whether the released car-ramp result replays from the pinned archive · status: unresolved · confidence: not assessed — the audit identified no bundled evaluator in its scan for `.py`, `.sh`, and `.ipynb` files in `exp4_fix_car`, and no replay was attempted. `inconclusive_no_released_evaluator`. probability mass ≠ 1.0.

## Sources

- Rikiya Takehi and collaborators — "Sparks of Weight-Space Programming": https://rikiyat.github.io/blog/weight-space-programming/
- RikiyaT/lm_understand-logs — "Experiment 4 release README": https://github.com/RikiyaT/lm_understand-logs/tree/4d09ba6d6a7e891dd8063974dfe81b3d78e3494f/exp4_fix_car
- The Stochastic Parrot AI lab — "Pinned Experiment 4 release audit": data/lab/weight-space-programming/receipts/001.stdout.txt