What was fixed, and why it matters
| Pull request | Fixed | Why it matters | Checked | Commit |
|---|---|---|---|---|
| keelo-nervous-system #29 | Diff the blocking check ids against the base, not the exit code (High) | A build that added new blocking eval failures passed as green whenever the base already had one red eval, and the fleet registered it with a checkmark. New failures now count as this build's, so a broken eval cannot ship behind an old one. | Verified: proof fails at head, passes on branch | 531d8d98 |
| keelo-nervous-system #29 | Treat a base with no manifest as unanswerable (High) | An agent's first eval claim was never scored when the base had no manifest; it was treated as already red. That case is now fatal and attributed to the build, so first evals get checked. | Verified: proof fails at head, passes on branch | f29a0559 |
| keelo-agent-factory #108 | Keep a blank or whitespace SLACK_TRIGGER_WORD falling back to DEFAULT_TRIGGER and map only the literal value none to "" (no run word) (High) | A blank SLACK_TRIGGER_WORD in a deployment's environment silently switched off `loop!`: the reply was filed as a team note and the held files were never processed. Blank now means the default `loop!`; only the literal `none` turns it off. | Verified: proof fails at head, passes on branch | 5d03144 |
| keelo-bug-sentinel #59 | Resolve the site-packages path before comparing it (High) | No release could install on the Amazon Linux 2023 boxes, because the venv reports a lib64 path the installer compared as text. The updater failed every 30 minutes and the heartbeat went stale. | Verified: proof fails at head, passes on branch | a47eb53 |
| keelo-bug-sentinel #59 | Stop the release venv writing bytecode (High) | One manual scan run as root wrote __pycache__ into the release; from then on every updater tick and runtime-ready check failed until someone deleted the files by hand. | Verified: proof fails at head, passes on branch | 906a67c |
| keelo-engineer #41 | Wait for the transcript before refusing a leave (High) | Kodak, the voice agent, refused "Kodak, leave" when the transcript had not caught up yet, and stayed in the meeting. It now waits for the transcript before deciding. | Verified: proof fails at head, passes on branch | 15436ee6 |
| kith-shoe-catalog-buysheet-agent #188 | Mirror the parked eyeball entry to disk and reload it at startup (Better option) | If the bot restarted while a buyer's eyeball check was parked, the buyer's next tap read "This check expired" and the work was lost. Parked entries now survive a restart. | Applied; diff read, suite run | 8f037c3 |
| kith-multibrand-buysheet-agent #167 | Leave the running service untouched on a deferral (High) | A deferred install restarted the running service, killing a buyer's job mid-run. The running service is now left untouched on a deferral. | Verified: proof fails at head, passes on branch | 54c1bac |
| kith-multibrand-buysheet-agent #167 | Pass RELEASE_BUCKET to the image-label check (High) | The image-label check never received the release bucket, so no OCI release could install. The bucket is now passed through. | Verified: proof fails at head, passes on branch | 6e34085 |
| kith-multibrand-buysheet-agent #167 | Generate the price table from a pinned LiteLLM pricing snapshot (Better option) | Seven files carried a hand-edited Opus price; one stale copy misreports cost. The table is now generated from a pinned pricing snapshot. | Applied; diff read, suite run | d6d20af |
Where Dorsal evidence shaped the fix
Every bug was found by reading the code and reproduced on the branch before it was graded. A Dorsal record was decisive where the fix follows what the source did, or where the item exists only because a source showed the better choice.
| Item | Dorsal source | What it settled |
|---|---|---|
| keelo-nervous-system #29, item 1 | NVIDIA/TensorRT-LLM #18766 and Qwen Code's review DESIGN.md | TensorRT-LLM's CI had hidden added regressions by exempting a whole case on a regressed base; Qwen Code compares the set of failures, not one bit. The fix diffs the failing check ids against the base instead of the exit code. |
| keelo-nervous-system #29, item 2 | Arize Phoenix evals README | Phoenix labels a missing input unmeasurable rather than a failure. The fix returns unanswerable for a base with no manifest instead of counting it as already red. |
| keelo-agent-factory #108, item 1 | Anthropic Python SDK #1726 and #1673; Claude Code #64710 | The SDK had shipped the same defect (a blank env value read as set) and repaired it by reading blank as unset at the read site; Claude Desktop had switched a feature off with an injected blank. The fix follows the SDK's repair: blank means the default, only the literal none turns the word off. |
| keelo-bug-sentinel #59, item 1 | NVIDIA/garak #1860 | garak hit the identical failure, a path check that fails where lib64 is a symlink to lib, and fixed it by resolving both sides. The fix does the same. |
| kith-shoe-catalog-buysheet-agent #188, item 1 | Cloudflare agents, human-in-the-loop docs; microsoft/agent-framework #7862 | This item is not a bug the reviewer reproduced; it exists because Cloudflare keeps a pending approval across a deploy and resumes on the user's tap, and a framework issue shows what happens when it cannot be restored. The upgrade mirrors the parked entry to disk and reloads it. |
| kith-multibrand-buysheet-agent #167, item 3 | ryoppippi/ccusage; OpenTelemetry GenAI conventions #443 | ccusage dropped its hand-edited price file and generates the table from a pinned pricing snapshot; the OpenTelemetry proposal records cost at the price in force at call time. The upgrade does the same. |
Each pull request
keelo-nervous-system PR #29
a red eval on a claimed eval is a broken build — and only when it is this build's red
1Diff the blocking check ids against the base, not the exit code
Broken eval announced finished.
Changeeval_check now collects the `BLOCK EVL-nn` ids from the build's `keelo eval check` output and from the base's (new `baseline_eval_blocks`, replacing `baseline_eval_rc`) and goes owed only when the build's ids are a non-empty subset of the base's; `_run_rc` returns full output so no BLOCK line is lost to the 600-character row cut, with `_run` and eval_check trimming to `DETAIL_TAIL` for display, and the four stubs in test_scaffold_and_gate.py were repointed at `baseline_eval_blocks`.
Fixed · commit 531d8d98 · agents/keelo-build/gate.py, agents/keelo-build/tests/test_scaffold_and_gate.py, agents/keelo-build/tests/test_review_eval_baseline.py
verified by a second agent: the proof fails at the PR head and passes on the branchtests/test_review_eval_baseline.py did not exist and was written from the finding's description; it runs the real `keelo eval check` from KEELO_FACTORY or ~/Downloads/keelo/keelo-agent-factory and skips without one.
2Treat a base with no manifest as unanswerable
First eval claim never scored.
Changebaseline_eval_blocks returns None when `tree / plan.MANIFEST` is not a file on the base, so a base with the agent directory but no manifest is unanswerable and the red stays this build's with the 'could not be checked' attribution.
Fixed · commit f29a0559 · agents/keelo-build/gate.py
verified by a second agent: the proof fails at the PR head and passes on the branchProof T was strengthened to assert the 'could not be checked' wording, since commit 531d8d98 by itself already turned this case fatal but attributed it to a green base.
git -C RUN_DIR/fix log -p bc5e22f9..dorsal-fix-pr29
git -C RUN_DIR/fix push -u origin dorsal-fix-pr29keelo-agent-factory PR #108
verbs and the run word are ThreadConfig knobs
1Keep a blank or whitespace SLACK_TRIGGER_WORD falling back to DEFAULT_TRIGGER and map only the literal value none to "" (no run word)
A loop deployment whose shell exports SLACK_TRIGGER_WORD= (blank) never runs from Slack: `loop!` is appended to the thread's docs/notes.md as a team note, the person gets a memo reaction, and the held files are never processed.
Changefrom_env strips SLACK_TRIGGER_WORD, falls back to loop! when blank, and maps only the literal value none to no run word.
Fixed · commit 5d03144 · harness/keelo_capabilities/slack_thread/listener.py, harness/keelo_capabilities/slack_thread/CLAUDE.md, harness/tests/test_slack_thread.py, harness/tests/test_pr108_defects.py, harness/tests/test_pr108_real_handler.py, docs/LOOPBOT.md, docs/INGREDIENT-LOOP.md
verified by a second agent: the proof fails at the PR head and passes on the branchThe two proof test files did not exist and were written under harness/tests/. The existing test that pinned blank = no run word (test_an_empty_trigger_means_no_run_word) now pins none and asserts blank = loop!. slack_thread/CLAUDE.md line 54 described the blank contract and was updated to name none.
git -C RUN_DIR/fix log -p cbd5f7d3..dorsal-fix-pr108
git -C RUN_DIR/fix push -u origin dorsal-fix-pr108keelo-bug-sentinel PR #59
Report Bug Sentinel runtime environment truthfully
1Resolve the site-packages path before comparing it
No release installs on the target boxes.
Changeverify_runtime now compares Path(observed['site']).resolve() with site_path.resolve(), so the venv's lib64 -> lib symlink alias matches the verified lib/ directory.
Fixed · commit a47eb53 · deploy/release.py, tests/review_al2023_lib64.sh, tests/review_fixture_release.py
verified by a second agent: the proof fails at the PR head and passes on the branchThe proof script and its fixture builder were missing from the worktree and were copied in from the review checkout. Two harness-only lines were added to the script so the minimal amazonlinux:2023 image matches a real box: dnf installs diffutils (installer calls cmp) and /etc/systemd/system is created (installer copies units there). Without those, the release.py fix alone moved the failure from the site-packages check to 'install: cannot create regular file /etc/systemd/system/bug-sentinel.service'.
2Stop the release venv writing bytecode
Updater and runtime evidence stop after one manual pass.
Changeruntime_path_file emits a second .pth line 'import sys; sys.dont_write_bytecode = True' so interpreters started from the release venv write no __pycache__.
Fixed · commit 906a67c · deploy/release.py, tests/test_review_bytecode.py
verified by a second agent: the proof fails at the PR head and passes on the branchThe reviewer's second test asserted the bug (expected __pycache__ and a ReleaseError), so it passed while the bug was present. It was rewritten as test_manual_root_run_writes_no_bytecode_and_release_stays_verifiable, which asserts no __pycache__ and a passing verify_runtime after a bare-environment import; it fails before the fix and passes after.
git -C RUN_DIR/fix log -p 326a1c83..dorsal-fix-pr59
git -C RUN_DIR/fix push -u origin dorsal-fix-pr59keelo-engineer PR #41
Land reviewed Kodak voice and transcription changes on main
1Wait for the transcript before refusing a leave
Kodak stays after being told to leave.
Change_leave re-reads state.heard up to ANCHOR_RETRIES times, sleeping ANCHOR_WAIT_S between reads, before refusing a leave.
Fixed · commit 15436ee6 · keelo_engineer/voice/server.py, tests/voice/test_review_pr41.py
verified by a second agent: the proof fails at the PR head and passes on the branchtests/voice/test_review_pr41.py did not exist in the worktree and was written there; pytest was not installed for any system Python, so a venv was created in the scratchpad with uv (python 3.12, pip install -e '.[dev]').
git -C RUN_DIR/fix log -p a6c5c122..dorsal-fix-pr41
git -C RUN_DIR/fix push -u origin dorsal-fix-pr41kith-shoe-catalog-buysheet-agent PR #188
Feat/eyeball preflight gate
1Mirror the parked eyeball entry to disk and reload it at startup
A bot restart while a card is parked turns the buyer's next tap into 'This check expired'; Cloudflare's agents SDK parks the approval durably so a deploy mid-prompt costs the user nothing.
ChangeEach parked eyeball entry is written to <RUNS_ROOT>/.pending_eyeball/<pending_id>.json on park/answer/ts-stamp, deleted on pop, and reloaded in build_app() under the original pending_id.
Applied · commit 8f037c3 · buysheet_v2/slack_bot/eyeball_persist.py, buysheet_v2/slack_bot/pending_eyeball.py, buysheet_v2/slack_bot/app.py, buysheet_v2/tests/test_eyeball_persist.py, buysheet_v2/tests/test_eyeball_gate.py
no proof to flip; the diff was read against what the upgrade namesFiles live under RUNS_ROOT/.pending_eyeball/ rather than a run_dir because the eyeball card parks before any run directory exists. TTL keeps counting across the restart via a wall-clock touched_at; files past the 1h TTL or unreadable are deleted at startup. test_eyeball_gate.py's autouse fixture now points jobs.RUNS_ROOT at tmp_path so the suite does not write into ~/buysheet_runs (which exists on this machine). No venv was present; one was built in the scratchpad (uv, Python 3.12, -e . plus Keelo-fashion-lens --no-deps, pytest, pypdf) to run the suite.
git -C RUN_DIR/fix log -p 858376af..dorsal-fix-pr188
git -C RUN_DIR/fix push -u origin dorsal-fix-pr188kith-multibrand-buysheet-agent PR #167
Emit truthful Multibrand telemetry and fail-closed releases
1Leave the running service untouched on a deferral
Buyer job killed mid-run.
Changerestore_runtime_snapshot takes a mode; on exit 75 main passes "deferred", which restores tags and files but skips both systemctl stop/start loops.
Fixed · commit 54c1bac · deploy/install-release.sh, buysheet_v2/tests/test_pr167_installer_review.py
verified by a second agent: the proof fails at the PR head and passes on the branchThe reviewer's test file did not exist in the worktree and its assertions encoded the bug (pass = bug present). It was rewritten under the same path and test names to assert the fixed behaviour, so it fails before and passes after. Two `l` loop variables renamed for ruff E741 (CI lint is diff-scoped).
2Pass RELEASE_BUCKET to the image-label check
No OCI release installs.
Changeprepare_oci_image passes "$RELEASE_BUCKET" (the variable perform_install sets) instead of the unset "$BUCKET".
Fixed · commit 6e34085 · deploy/install-release.sh, buysheet_v2/tests/test_pr167_installer_review.py
verified by a second agent: the proof fails at the PR head and passes on the branchOne-token change at deploy/install-release.sh:839. Under macOS bash 3.2 the unset variable does not abort, so the proof needs bash 5 on PATH.
3Generate the price table from a pinned LiteLLM pricing snapshot
Seven files in this PR change the Opus rate by hand; ccusage stopped trusting a hand-edited pricing file and generates its table from a pinned LiteLLM snapshot at build time.
ChangeNew generator writes buysheet_v2/pricing_table.py from a pinned LiteLLM commit; anthropic_client.py and gemini_telemetry.py read rates and PRICING_VERSION from it.
Applied · commit d6d20af · buysheet_v2/tools/gen_pricing_table.py, buysheet_v2/pricing_table.py, buysheet_v2/anthropic_client.py, buysheet_v2/gemini_telemetry.py, buysheet_v2/tests/test_pricing_table.py, buysheet_v2/tests/test_anthropic_client_telemetry.py, buysheet_v2/tests/test_gemini_telemetry.py
no proof to flip; the diff was read against what the upgrade namesIts only check is the existing suite. Two test edits were required: version-string literals in test_anthropic_client_telemetry.py now reference pricing_table constants, and test_gemini_telemetry's mixed-provider test now asserts a run-level total because both providers share one pricing identity (runs.py aggregates cost when every call has the same pricing_version). extract_gemini.py:87 carries a third hand-typed table outside the named scope and was left alone. Regenerate with: python -m buysheet_v2.tools.gen_pricing_table --commit <sha>.
git -C RUN_DIR/fix log -p 2046bf75..dorsal-fix-pr167
git -C RUN_DIR/fix push -u origin dorsal-fix-pr167