The session review checklist
Published July 31, 2026 · by the AQ team
A session review checklist is a set of reviewer questions that covers the run that produced AI-generated code, not just the diff it left behind. This one has five layers: the brief the agent was given, the corrections it received mid-run, the paths it tried and abandoned, the warnings the operator moved past, and the behavior of the running result. Each layer below is three to five questions with the reason it matters. It works with any terminal coding agent, takes ten to fifteen minutes for a typical pull request, and assumes nothing beyond a saved prompt and a retrievable transcript.
Why a checklist at all: most agent code currently merges with one set of eyes on it. A July 2026 LeadDev analysis of 25,264 agent-generated pull requests across 2,361 popular GitHub repositories found that in 79 percent of agentic PRs the same developer both reviewed and modified the agent's contribution, and only about one in eight workflows involved multiple humans. A reviewer who did not run the session needs a script for what to look at; that script is this page. The argument for why the diff alone is not enough lives in the self-review problem, the term itself is defined at session review, and the surrounding habits (preserving briefs, saving transcripts) are covered in how to review an AI coding session.
What you need before starting
- The original prompt, verbatim. Ideally pasted into the PR description, including any mid-run follow-ups that changed scope.
- The session transcript. A saved scrollback, a CLI export, or the live session itself. If the agent ran in tmux, one command recovers it: tmux capture-pane -p -S - redirected to a file. Claude Code's /export command and Codex CLI's session files under ~/.codex/sessions/ work too, as of July 2026.
- A way to run the result. The branch checked out, a preview environment, or a teammate's screen. Layer five is not optional.
Layer 1: the brief
Read the prompt before the code. The most characteristic agent failure is plausible code that does something subtly different from the request, and only the brief exposes it.
- Do you have the actual prompt, or a paraphrase? A summary written after the fact inherits the operator's expectation bias, which is the thing a second reviewer exists to remove.
- Does the diff implement what was asked, or a plausible neighbor of it? Read the brief, predict the change, then compare; drift between prediction and diff is the highest-signal finding in the whole review.
- Did mid-run follow-ups change the scope? A "skip the migration for now" halfway through the session explains an absence the diff cannot; without it a reviewer flags the gap or wrongly assumes it was forgotten.
- Was the ask itself right? Agents almost never push back on a bad ticket, so the reviewer is the first person positioned to say the task should not have been done this way at all.
Layer 2: the corrections
Skim the transcript for the places the operator had to redirect the agent. You are not rereading the session; you are collecting coordinates.
- Where did the operator say some version of "no, not like that"? Every correction marks a spot where the agent's first instinct was wrong, and a correction fixes the instance, not the instinct.
- Is the code near each correction sound? Concentrate diff-reading time there; the surrounding logic was written by the same instinct that just got corrected.
- Was any correction applied only partially? Agents routinely fix the named occurrence and miss the pattern's other call sites.
- Did the same correction recur? Three redirections on one point means the agent never really understood it; treat that area as unreviewed human code from an unfamiliar contributor.
Layer 3: the abandoned paths
Dead ends are signal. If the agent tried an approach and backed out, it found that part of the codebase confusing, and confusing code is where subtle breakage lives.
- What did the agent try and abandon? The retreat tells you which files it half-understood; read those parts of the diff with the least charity.
- Is the retreat clean? Leftover imports, orphaned helpers, and half-renamed symbols are the classic residue of an abandoned approach surviving into the final diff.
- Did a dead end touch files the final approach never needed? Changes with no connection to the shipped design are usually leftovers, not intent.
- Do dependencies added for an abandoned path still ship? A package installed for attempt one and unused by attempt three is dead weight at best; verify anything unfamiliar actually exists and is the intended project.
Layer 4: the overrides
Look for the moments the agent flagged uncertainty, suggested a test, or raised a warning, and a human moved past it. These are the cheapest review targets in the transcript: the agent has already told you where it was unsure.
- Where did the agent hedge, warn, or ask? Phrases like "note that this assumes" or "you may want to verify" are the agent marking its own low-confidence output.
- Which of those did the operator act on, and which were waved past? Overrides made under deadline pressure are exactly what a second opinion is for.
- Did the agent suggest a test that was never written? A declined test suggestion is a known gap with a written confession attached.
- Would you have made the same call? If an override looks wrong to you, that single disagreement usually justifies the entire review.
Layer 5: the running behavior
Run the result. Most plausible-but-wrong agent code fails within the first minute of someone other than the operator actually exercising it, which makes this the highest-yield minute in the review.
- Has anyone other than the operator run this? The operator already saw it "work" once, under the same expectations that shaped the prompt.
- Does the happy path survive being clicked, not just read? Code review verifies plausibility; execution verifies behavior.
- Do the unhappy paths behave? Empty states, error responses, a reload mid-flow: agents overwhelmingly optimize for the demonstrated path.
- Does the behavior match the brief, not just the diff? This closes the loop with layer one; the feature can be implemented exactly as coded and still not be what was asked for.
The checklist at a glance
| Layer | What it catches | Fastest check |
|---|---|---|
| The brief | Plausible code that answers a different question | Read the prompt, predict the diff, compare |
| The corrections | Surviving code written by a corrected instinct | Search the transcript for redirections; read nearby diff closely |
| The abandoned paths | Residue: dead imports, orphaned helpers, stray dependencies | Match every diff hunk to the final approach |
| The overrides | Known risks someone decided not to care about | Find the agent's own warnings; judge each call |
| The running behavior | Code that reads correctly and runs wrong | Exercise the result for one minute as a stranger |
Running it in practice
Budget ten to fifteen minutes: one on the brief, three or four skimming the transcript for corrections and overrides, one or two exercising the result, and the remainder on the diff with everything the session told you about where to look. The transcript work is targeted scanning, not a replay; redirections and warnings stand out visually in scrollback. If a layer is impossible because its input is gone (no saved prompt, no transcript), say so in the review rather than silently skipping it: "reviewed without session context" is honest and tells the team which pipeline to fix. And keep the whole thing lightweight on small changes; a checklist that takes longer than the diff deserves gets abandoned by Friday.
A closing note on tooling
Everything above works with a text file and discipline. The recurring cost is retrieval: getting the brief, the transcript, and a runnable result in front of a second person. AQ is the multiplayer coding harness where engineering teams run AI coding agents like Claude Code and Codex together: shared live terminals, a code editor, and app previews, in your own cloud. Agents run as real CLIs in persistent tmux sessions on your team's VM, so the transcript a reviewer needs is the workspace's live terminal and scrollback, the diff is in the workspace editor, and the running result is that workspace's live preview, one link away; the PRs the agent opens are tracked on the same workspace. The checklist does not change. The setup time to run it approaches zero.
Frequently asked questions
Do I need to read the entire transcript to use this checklist?
No. Layers two through four are targeted scanning, not a replay: you are searching the scrollback for redirections, backed-out approaches, and the agent's own hedging language, all of which stand out visually. For a typical session that is three or four minutes of skimming. The only parts read closely are the diff regions those scan hits point at.
Who should run this checklist, the operator or a second person?
A second person, wherever possible. The operator can and should self-check against it, but the checklist's main job is de-biasing, and the operator carries the exact expectations that hide the characteristic failure (plausible code that does something subtly different from the ask). Teams that want it to stick make a non-operator reviewer the default on agent-assisted PRs, the same way protected branches already require an approver.
What if the session transcript is already gone?
Run the layers that survive. Ask the operator for the prompt and any scope-changing corrections and paste them into the PR (layer one, most of two), and have a non-operator exercise the result (layer five). Note explicitly that layers three and four were unreviewable. Then fix retention going forward: tmux capture-pane, the script utility, or the CLI's own export, and note that Claude Code's local transcripts are cleaned up after 30 days by default as of July 2026, so an export beats relying on history.
Can any of this checklist be automated?
Partially. Linters and automated PR reviewers cover part of layer three (unused imports, dead code, unknown packages) and CI covers whatever tests exist. But layers one, two, and four live in the prompt and the transcript, which diff-reading tools never see, and layer five is by definition a human exercising the result. Automate the code layer so the human minutes go where only humans can look.
How is this different from a generic AI code review checklist?
Generic checklists for AI-generated code (check imports resolve, look for hallucinated APIs, verify error handling) operate on the diff. This checklist operates on the session: the brief, the corrections, the abandoned paths, and the overrides are all invisible in the diff no matter how carefully it is read. The two compose: run a diff-level checklist as usual, and use the session layers to decide where the diff deserves the closest reading. The diff-level mechanics are covered in reviewing pull requests from AI agents.