Rolling out session review: a playbook for engineering leaders

Published August 1, 2026 · by the AQ team

Rolling out session review means making the AI agent session itself (the brief, the mid-run corrections, the abandoned paths, the overrides, the running behavior) a reviewable artifact, and making a second person the default on agent pull requests, as written team policy rather than individual habit. A workable rollout has three components: a rule for when a second person is required, the artifacts every agent session must leave behind, and a time budget small enough to survive a deadline week. The term itself is defined at session review; the argument for why the diff alone is not enough lives in the self-review problem.

Why this needs a policy, not a suggestion

Left to individual habit, agent work reviews itself. A July 2026 LeadDev analysis of 25,264 agent-generated pull requests across 2,361 popular GitHub repositories found that in 79 percent of agentic PRs the same developer both reviewed and modified the agent's contribution, and only about one in eight workflows involved multiple humans. Nobody decided agent code should skip independent review; it happened by default, because the session lived in one person's terminal and the diff was the only artifact anyone else could see.

That is why a memo saying "please review agent PRs carefully" changes nothing. The default is structural, so the fix has to be structural: a policy that names the required reviewer and the required artifacts, enforced like your branch protection rules. The same LeadDev piece names the leader-shaped fix: team-level learning rather than solo experimentation, and tracking quality rather than code volume.

Write the second-person rule first

The core of the policy is one sentence: agent pull requests get a second person by default, and that person gets the session, not just the diff. The design work is deciding where "by default" hardens into "always" and where it relaxes. A risk tier does that cleanly:

• Always, no exceptions: authentication and authorization, payments and billing, data migrations or deletions, infrastructure and CI configuration, and anything touching a public API contract. These are the changes where plausible-but-wrong is most expensive, so the second reviewer is a hard merge requirement.
• Default, with an opt-out the author must state: everything else headed to production. The author can write "self-reviewed: low-risk, test-only change" in the PR description; the opt-out is visible and auditable, and silence is not an opt-out.
• Exempt: prototypes, spikes, and throwaway branches that will never merge to main. Review theater on disposable code teaches people the policy is theater everywhere.

Key the tier to what the change touches, not to who or what wrote it. A separate second-class lane for "AI code" invites relabeling games; a lane keyed to blast radius does not.

Require artifacts, not effort

A policy that demands "thorough review" is unenforceable. A policy that demands two artifacts is checkable in seconds:

• The brief. The original instruction given to the agent, pasted into the PR description or linked from it. The most common serious agent failure is a correct-looking implementation of the wrong interpretation, catchable only by someone holding the actual request.
• The session record. The transcript or terminal scrollback of the run, retrievable by the reviewer. How to capture this with any terminal agent (tmux scrollback, script, or the CLI's own export) is covered step by step in how to review an AI coding session.

With those two in hand, the reviewer works through the five layers of the session: the brief the agent was given, the corrections it received mid-run, the paths it tried and abandoned, the warnings the operator moved past, and the behavior of the running result. You do not need to invent the reviewer's script; the session review checklist is a ready-made set of questions for each layer, and the single most useful thing to link from your policy doc.

Keep it light enough to survive deadlines

Every review practice gets stress-tested by its first crunch; heavy ones fail quietly, with people approving without reading. Google's public engineering practices are blunt about the mechanism: one business day is the maximum acceptable time to respond to a review request, because slow review creates pressure to cut corners and stalls the whole team's velocity. Session review has to live inside the same constraint. Three design choices keep it there:

• Budget ten to fifteen minutes, and say so in the policy. The checklist above fits that budget for a typical PR. A reviewer who knows the expected cost is fifteen minutes will start today; one who suspects it is ninety will find a reason to defer.
• Let the session direct the effort. The scrollback shows where the agent struggled, got corrected, or retried. Those files get the close read; the parts the agent sailed through get a skim. That is what makes session review cheaper than diff-only review, not more expensive: the transcript is a map of where the risk is.
• Make it asynchronous by default. The reviewer reads the brief and the scrollback on their own schedule, like any other review. Watching live is valuable for mentoring, but the policy should never require two calendars to align before a merge.

Measure quality, not code volume

A working rollout will not show up as more merged PRs, and measuring merged PRs accidentally punishes it. The LeadDev analysis argues for tracking quality over volume, and the broader delivery data says the same thing: the 2025 DORA report found AI adoption positively associated with delivery throughput but negatively associated with delivery stability, as of its September 2025 publication. Volume is the thing AI already inflates; stability is the thing your review practice exists to protect. Four numbers tell you whether it is working:

• Second-human rate: the share of agent-assisted PRs where someone other than the author reviewed or modified the change. The direct counter to the 79 percent figure, and the rollout's headline metric.
• Rework rate: how often agent-assisted changes are reverted or substantially rewritten within 30 days. The quality signal the practice should move.
• Time to first review: if this climbs past a business day, the practice is too heavy and rubber-stamping is coming. Treat it as an early-warning gauge, not a leaderboard.
• Opt-out rate: how often the self-review opt-out is used, and by whom; a drifting rate is the policy quietly dissolving.

Three failure modes to design against

Rubber-stamping. The second approval becomes a checkbox click within seconds of the request. Defense: require the artifacts, not just the approval; an approval on a PR with no brief and no session record is visibly hollow. Spot-check a few approvals a month by asking the reviewer one question about the session.

Review theater. The policy exists, the PR template has the fields, and nobody actually reads the transcripts. Defense: keep the budget honest (fifteen minutes, checklist linked) and have leads do session reviews themselves, visibly. Practices propagate by imitation, not by policy doc.

Blocking on absent owners. The rule says "the code owner must review the session," the owner is on vacation, and by Thursday the team has learned to route around the policy. Defense: any second engineer qualifies by default; owners are required only in the always-tier, and the always-tier names at least two people per area.

Where AQ fits

AQ is the multiplayer coding harness where engineering teams run AI coding agents like Claude Code and Codex together: shared live terminals, a code editor, and app previews, in your own cloud. For a session-review rollout, what matters is that the artifacts the policy requires stop being homework. Agents run as real CLIs (Claude Code, Codex, Cursor Agent, Kimi, Grok, or plain shells) in persistent tmux sessions on your team's VM, streamed live to the browser, so the session record is simply the workspace: a reviewer opens it and reads the same live terminal and scrollback the operator saw, checks the diff in the editor, and clicks the running result in the workspace preview. Comments pinned on the preview can be sent to the agent as prompts, the PRs the agent opens are tracked on the workspace that produced them, and sessions survive a closed laptop, so the review can happen hours after the run.

The policy layers map onto workspace visibility, which stays owner-managed: workspaces are team-visible by default or private and shared with specific people, so "the reviewer gets the session" is a link, not a file-collection exercise. Teams driving work from Linear can label an issue ai-task and a workspace appears with ownership following the assignee, which keeps "who reviews this" adjacent to "whose issue is this." Pricing is two plans: Free (the full product, bring your own VM, you pay your cloud provider) and Team at $100 per user per month (promotional; standard $200, billed monthly) with a dedicated always-on AQ-managed VM in its own isolated network and your rate locked for your first 12 months.

The policy is still yours to write, and the manual version of this practice works with any terminal agent today. What a shared harness changes is the rollout cost: the artifacts exist by default, so the policy only has to say who looks at them.