The React fix that passed every check and changed nothing
A wrong fix typechecked and passed genie-react verify because the click hit a disabled menu item and no row rendered. One render min 1 claim turned the pass into a fail.
The fix passed because nothing rendered. A claim that limits renders, such as
render OrderRow only props.order,mount, also holds when zero rows render, so a run whose click
missed the app looks clean. Adding render OrderRow min 1 made the same run fail with exit code 1.
This happened in a real run on the Meridian Ops dashboard in this repo
(apps/realworld-dashboard). The commands and untrimmed output are in the
evidence file.
The bug and a fix that looked right
Putting one order on hold re-renders every visible OrderRow, because the table's
onStatusChange callback lists rows as a dependency.
Your AI agent said 'fixed' walks through that bug.
On unchanged code, this run failed with exit code 1 (trimmed):
fail vfy_63b3257e-02dc-4641-9eb0-d68f05c5214a "put one order on hold" 14 commits, quiet after 917 ms (react, query, router, network)
1 fail render OrderRow only props.order,mount {renders:24,events:24,violatingEvents:24,causes:[props.onStatusChange,props.order]}
2 pass boundaries clean {caughtDuringWindow:0,caughtAtEnd:0}Then came a fix that an agent could write after reading "remove rows from the dependencies":
--- a/apps/realworld-dashboard/src/features/orders/orders-table.tsx
+++ b/apps/realworld-dashboard/src/features/orders/orders-table.tsx
+ const ordersById = useMemo(() => new Map(rows.map((row) => [row.id, row])), [rows])
const onStatusChange = useCallback(
(id: string, status: OrderSummary['status']) => {
- const order = rows.find((row) => row.id === id)
+ const order = ordersById.get(id)
if (order) changeStatus({ id, number: order.number, status })
},
- [rows, changeStatus],
+ [ordersById, changeStatus],
)rows is gone from the callback's dependencies, and npx tsc --noEmit -p . printed
typecheck ok. But ordersById is rebuilt every time rows changes, so the callback still
changes with every data update. The fix does nothing.
The rerun passed with zero renders
The agent reran the exact commands from the first run:
npx genie-react verify begin --session upd-b --name 'put one order on hold' \
--claim 'render OrderRow only props.order,mount' --claim 'boundaries clean'
agent-browser --session upd-b click 'button[aria-label="Actions for SO-105999"]'
agent-browser --session upd-b find role menuitem click --name 'Put on hold'
agent-browser --session upd-b wait 1500
npx genie-react verify end vfy_e7e11896-11d9-4dd4-94df-8465e8987c3b --session upd-bBoth clicks printed ✓ Done. The verdict, trimmed:
pass vfy_e7e11896-11d9-4dd4-94df-8465e8987c3b "put one order on hold" 9 commits, quiet after 1027 ms (react, query, router, network)
1 pass render OrderRow only props.order,mount {renders:0,events:0,violatingEvents:0,causes:[]}
2 pass boundaries clean {caughtDuringWindow:0,caughtAtEnd:0}Exit code 0. Both claims passed and the code typechecked. The giveaway is renders:0. Before the change, the same claim saw 24 renders.
The click never reached the order
The first run had already put SO-105999 on hold, and the row menu disables Put on hold for an order that is on hold. A snapshot after the rerun shows the menu still open (trimmed):
agent-browser --session upd-b snapshot -i | grep -A4 'Actions for SO-105999' - gridcell "Actions for SO-105999" [ref=e69]
- button "Actions for SO-105999" [expanded=true, ref=e195]
- menuitem "Put on hold" [disabled, ref=e206]
- menuitem "Mark shipped" [ref=e207]
- menuitem "Cancel order" [ref=e208]agent-browser reported the click on the disabled item as ✓ Done. No status change happened, so
no row had a reason to render.
Genie has a guard for a window where nothing happened. It refuses with no_activity when the
window has no React commit at all, as the refusal list
describes. This window had 9, from opening the menu and from a clock on the page, so the guard did
not apply.
On this page the guard can't apply even with no interaction. Ending a window without touching the app also passed, with exit code 0 (trimmed):
pass vfy_f897206c-6cd6-48ed-bb2a-4b18de4e9af3 "put one order on hold" 7 commits, quiet after 1174 ms (react, query, router, network)
1 pass render OrderRow only props.order,mount {renders:0,events:0,violatingEvents:0,causes:[]}verify show with --section raw:react_get_renders names the source of those 7 commits. Selected
fields:
{name:SyncStatus,mountId:mount:13,renders:7,updates:7,causes:[{kind:state,evidence:exact,name:"state[0]"}],necessity:necessary,necessityReason:dom-changed,source:{location:/src/features/orders/orders-page.tsx:50:11,role:element-callsite,ownership:app}}SyncStatus shows "Updated N seconds ago" and sets state once a second. A page with a ticking
clock commits on its own, so a verify window there is never empty for long.
A min 1 claim turns the miss into a fail
What catches this is a positive control, a claim that only the interaction can make true. Here,
at least one OrderRow has to render. Same wrong fix, same disabled menu item, one more claim:
npx genie-react verify begin --session upd-b --name 'put one order on hold' \
--claim 'render OrderRow only props.order,mount' \
--claim 'render OrderRow min 1' \
--claim 'boundaries clean'After the same two clicks, verify end printed (trimmed):
fail vfy_7b4fb2bb-b4e6-4a27-966b-a33b997ea461 "put one order on hold" 9 commits, quiet after 1036 ms (react, query, router, network)
1 pass render OrderRow only props.order,mount {renders:0,events:0,violatingEvents:0,causes:[]}
2 fail render OrderRow min 1 {renders:0,mounts:0,updates:0,remounts:0,semantics:exact,instances:12,activeInstances:12}
3 pass boundaries clean {caughtDuringWindow:0,caughtAtEnd:0}Exit code 1. Claim 1 still passes, because it is still true. Claim 2 explains why claim 1 proves nothing. Twelve rows were mounted and none rendered.
On a row that can go on hold, the fix fails
The agent reset the demo data with npx genie-react call app_reset_backend '{}' --session upd-b,
reopened the page, and ran the three claims again. This time the click reached the order (trimmed):
fail vfy_fc65520e-3289-47b1-ab0b-fcae777c96a6 "put one order on hold" 16 commits, quiet after 916 ms (react, query, router, network)
1 fail render OrderRow only props.order,mount {renders:24,events:24,violatingEvents:24,causes:[props.onStatusChange,props.order]}
2 pass render OrderRow min 1 {renders:24,mounts:0,updates:24,remounts:0,semantics:exact,instances:12,activeInstances:12}
3 pass boundaries clean {caughtDuringWindow:0,caughtAtEnd:0}Exit code 1. Twenty-four renders, the same count as unchanged code, still caused by
props.onStatusChange. The fix that passed changed nothing. The fix that works passes the order
object to the callback and drops rows entirely. The
other article shows it passing with 2 renders.
Refused and inconclusive are not a pass either
verify has two more verdicts. Your agent should treat both as "not proven".
Reloading the page between begin and end gave refused with exit code 4. Each refusal prints
the command that starts a clean window (trimmed to claim 1):
refused vfy_6803f668-9ff9-41ba-9757-58c785ca9778 "put one order on hold"
1 refused render OrderRow only props.order,mount session_changed: The document closed or reloaded during the window.
fix: $ npx genie-react verify begin --session upd-b --name 'put one order on hold' --claim 'render OrderRow only props.order,mount' --claim 'render OrderRow min 1' --claim 'boundaries clean'The $ line is runnable and repeats the original claims, so the rerun checks the same thing. It
starts with npx, the runner this run used.
Opening the search palette with effect CommandPalette max 1 gave inconclusive with exit code 5
(trimmed):
inconclusive vfy_f12f14af-ba52-456c-9216-611b4af4a15f "open search" 7 commits, quiet after 800 ms (react, query, router, network)
1 inconclusive effect CommandPalette max 1 strict_mode_replay: The raw count exceeds 1 only by StrictMode's development replay of a fresh mount, which a production build does not run.
observed: {effects:[{effectIndex:0,runs:1,rawRuns:2},{effectIndex:1,runs:1,rawRuns:2}]}
2 pass render CommandPalette min 1 {renders:1,mounts:1,updates:0,remounts:0,semantics:exact,instances:1,activeInstances:1}Each effect ran twice. One of those runs is StrictMode replaying the fresh mount in development.
The evidence is complete, but it cannot decide a limit of 1 for a production build. An
inconclusive verdict prints no $ fix. The agent has to change the claim or report what it saw.
When a positive control is not enough
min 1proves the component rendered, not that the interaction caused it. Pick a target that only your interaction changes.SyncStatuswould be a bad control on this page, because it renders every second.- Genie does not check the screen. Every verdict above carried
! ui-outcome-not-observed: Genie does not check the screen; confirm it with the UI driver. Here, a driver check of the row's status would have shown it was already On hold. - A verdict covers one interaction on one tab. It does not tell you the other two menu items behave the same.
The claim forms and every refusal reason are in Verify a fix and Read evidence.
Your AI agent said 'fixed'. Make it prove it with an exit code
A real run of genie-react verify. Putting one order on hold re-rendered 12 rows twice, the verdict failed with exit 1, and after the fix it passed with exit 0.
We planted 15 bugs in a legacy React and Redux app. Here's what an agent could prove
Genie's evidence proved 11 of 15 seeded bugs and showed only the symptom for 4. verify turned fail into pass for 11 fixes and had no claim for the other 4.