# Software Factory Levels: From Autocomplete to the Dark Factory

> The levels of AI coding autonomy, from autocomplete to a lights-out dark factory: what each level requires, how it fails, and how to check your repo's readiness. AQ is a platform for running a team software factory at levels 2 through 4: AI coding agents like Claude Code and Codex work in parallel, each in its own branch, a second agent reviews, and your team approves the plan and the merge together.

Published November 13, 2026 · by the AQ team · Canonical: https://aq.dev/guides/software-factory-levels-from-autocomplete-to-dark-factory/

A dark software factory, also called a lights-out software factory, is a setup where coding agents turn a specification into shipped software with no person writing or reviewing the code along the way; people write the specs and the checks, and machines do the rest. As of November 2026 most teams can run one level below that: many agents working in parallel from a task queue, each in its own isolated environment, with a person approving plans and deciding what merges. Before people step out of the loop, a team needs checks that would catch what a reviewer catches, an automatic rollback, hard spending limits, and a record of every action.

**AQ is a platform for running a team software factory at levels 2 through 4: AI coding agents like Claude Code and Codex work in parallel, each in its own branch, a second agent reviews, and your team approves the plan and the merge together.** AQ runs the levels below the dark factory today: each task in its own git worktree, a Lead agent whose plan waits for approval, review routed to a different vendor than the author where one is connected, routines on schedules and events, and a person deciding what merges.

## Where the five levels come from

The ladder most people cite is Dan Shapiro's essay ["The Five Levels: from Spicy Autocomplete to the Dark Factory"](https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/), published on his blog on January 23, 2026 and discussed on [Hacker News](https://news.ycombinator.com/item?id=46739117) the same week. Shapiro, the CEO of Glowforge, modeled it on the NHTSA's levels of driving automation, with a car analogy for each rung, and took the top rung's name from FANUC's robot-run factory. He writes that most developers who call themselves AI-native sit in the middle of the ladder and that almost everyone tops out at the level where a person manages several agents and reviews their diffs.

The most cited team claiming the top rung is StrongDM. In ["Software Factories And The Agentic Moment"](https://factory.strongdm.ai/) (February 6, 2026), CTO Justin McCarthy describes rules that code must not be written or reviewed by humans, with end-to-end scenarios kept outside the codebase as a holdout set and behavioral clones of third-party services (Okta, Jira, Slack, Google Docs and others) so validation can run far beyond production rate limits. Those are the company's own claims about its internal process. Skeptics in the Hacker News thread called levels 4 and 5 unproven outside well-funded, controlled settings, and Simon Willison described the dark factory idea there as basically unproven.

The definitions below follow Shapiro's numbering so readers can map between them, but the wording and the requirements are ours. For the broader term and how to build one, see [what is a software factory](https://aq.dev/software-factory/).

## The levels, what each requires, and how to check readiness

| Level | Who does what | What must exist | Failure mode | How to check readiness |
| --- | --- | --- | --- | --- |
| **0. Manual** | A person writes every line. AI answers questions or suggests a token now and then. | Nothing beyond a normal repo and CI. | None specific to AI. | Not applicable. |
| **1. Assisted** | A person writes the important code and hands the agent small, bounded tasks: a unit test, a docstring, a rename. | A test suite the person runs before committing; a linter. | Accepted suggestions nobody read closely, such as a test that asserts the bug. | Run the suite locally in one command. If that takes more than a few minutes or needs tribal knowledge, fix it first. |
| **2. Paired** | A person and one agent work in the same session. The agent writes most of the code; the person reads every change before it is committed. | Fast tests, a clear AGENTS.md or CLAUDE.md, and a branch per change. | The person approves changes faster than they read them, and quality drops without anyone noticing. | Count reverts and hotfixes of agent-written commits over the last 90 days. |
| **3. Supervised fleet** | Several agents run in parallel on separate tasks. A person assigns work, approves plans and reviews every diff before merge. | One isolated environment per agent (a worktree or VM), CI that runs on every branch, branch protection on main, and a review queue someone owns. | Review becomes the bottleneck and concentrates on one person, who starts skimming. | Measure review time per pull request and how many people review agent work. One reviewer for a whole team's agents is a warning. |
| **4. Spec-driven** | People write specs, acceptance criteria and agent instructions, approve the plan, then leave. Agents build, test and open pull requests. People check results and review a sample or the risky paths. | Acceptance tests derived from the spec, preview environments, per-task and monthly spend caps, a run record for each agent, and fast rollback. | Specs are vague, so agents build the wrong thing confidently and the tests, written by the same agents, agree with them. | Give an agent three recent tickets on a branch, with only the ticket text. Check whether it finishes each and whether your tests reject a deliberately wrong solution. |
| **5. Dark factory** | Specs and scenarios go in; merged, deployed software comes out. No person writes or reads the code. | Holdout scenarios the agents cannot see or edit, test doubles of every external service, automatic rollback on bad health signals, hard spend limits, an audit trail of every action, and a person on call for incidents. | Agents satisfy the checks while missing the intent, and errors compound across releases until a human has to read code nobody understands. | Run the factory on a low-risk service for a month with human review still on, and record every problem a reviewer caught that the scenarios missed. A month with none is one necessary piece of evidence; it does not prove the repository is safe for level 5, so keep human review for consequential or uncertain changes. |

Two notes on reading the table. The levels describe a task type, so one team can run dependency bumps at level 4 and payment code at level 2. And each level moves human judgment to a different place: at level 3 it is in reading diffs, at level 4 it is in writing specs and acceptance tests, and at level 5 it is in designing the scenarios and owning incidents.

## What stays human even at the top

Every published framework for agent safety keeps people in charge of a short list of decisions, even when the code itself is machine-written and machine-checked.

- **Merge policy for risky paths.** GitHub's [protected branches](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches) can require approving reviews from people with write access or from a designated code owner before anything merges. That rule is enforced by the host, so it holds even when an agent has push credentials. Check that bypass is turned off for admins, since protection exempts them by default.
- **High-impact actions and permissions.** The OWASP Top 10 for LLM applications lists [excessive agency](https://genai.owasp.org/llmrisk/llm062025-excessive-agency/) as a risk and recommends minimal tools, minimal permissions and a person approving high-impact actions before they happen. Production credentials, data deletion and money movement belong in that category.
- **Approving what an agent runs.** Anthropic's [Claude Code security documentation](https://code.claude.com/docs/en/security) says the user is responsible for reviewing proposed code and commands before approving them, and describes an auto mode in which a classifier model reviews actions instead. Choosing which of those applies, and where, is a human decision.
- **Incident response.** Google's SRE book chapter on [managing incidents](https://sre.google/sre-book/managing-incidents/) assigns an incident commander who holds the overall state and hands off command explicitly. A factory that ships on its own still needs a named person who can stop it, roll back and talk to customers.
- **Secure development practice.** NIST's [Secure Software Development Framework](https://csrc.nist.gov/pubs/sp/800/218/final) (SP 800-218) sets out practices any development process should meet, regardless of who writes the code. Someone has to own conformance.

The [software factory security guide](https://aq.dev/guides/software-factory-security-permissions-secrets-review-gates/) covers permissions, secrets and review gates in detail.

## Where AQ fits

AQ is a platform for running a team software factory at levels 2 through 4: AI coding agents like Claude Code and Codex work in parallel, each in its own branch, a second agent reviews, and your team approves the plan and the merge together.

AQ is where teams put AI agents to work turning specifications into reviewed pull requests, using the agents and subscriptions they already have on private cloud machines of their own. It supports levels 2 through 4 today, and gives a team the parts that level 5 builds on.

- **Isolated work items (levels 2 and 3).** Each work item gets its own git worktree and branch on a dedicated cloud machine or your own Mac, so parallel agents never share a checkout. Work can start from the composer, a Linear label, the API or a bot.
- **Crews with a plan gate (levels 3 and 4).** A [crew](https://aq.dev/docs/crews/) has a Lead agent that plans and delegates build, test, review and research to other agents. With the plan gate on, the Lead proposes its steps and waits for you to approve, edit, or run them in one thread, and no delegated role that can edit code starts until you approve. The gate covers delegation only: the Lead's own thread can still edit files in its worktree before approval.
- **Independent review.** When AQ's Auto chooses the reviewer, it prefers a different vendor and account than the one that wrote the code. Merge in the [Changes panel](https://aq.dev/docs/changes/) is refused, with the reason in words, while checks are pending or failing or while the review is older than the head commit.
- **Routines (level 4).** [Bots](https://aq.dev/docs/bots/) run routines on schedules, monthly dates, intervals, Linear labels, Gmail, Google Calendar, GitHub events, failing checks and review comments, and webhooks, and can start coding work items. Each run records why it woke, what it produced and what it cost. A bot's monthly cap covers the AQ credits its routines spend, and an optional ceiling stops a single run.
- **API and MCP.** A REST API, signed webhooks and a hosted MCP server are on every plan, Free included, so an outside system or agent can create work, send messages and read results.

What AQ does not do on its own: it does not provide production deployment, so code reaches production only through your own CI/CD. It adds no automatic merge step, and the team owns the specs, the test quality and the decision to merge. Agents run with each person's own GitHub access, so an agent can run any git command those credentials allow; branch protection on your host is what stops an unapproved merge. AQ also does not claim to run a level 5 factory for you. The [guide to building a software factory with Claude Code and Codex](https://aq.dev/guides/build-a-software-factory-with-claude-code-and-codex/) walks through a level 3 to level 4 setup step by step.

## How to check this in AQ

Two things this page says about isolation and review can be checked in any AQ work item. These checks show how one work item is isolated and gated. They say nothing about whether any repository is ready for level 4 or level 5.

- **Its own checkout and branch.** Each work item runs in its own git worktree on its own branch, and other work items on the same machine each have theirs. You can run the same check on any machine with `git worktree list`, which prints each checkout and the branch it holds. Edits made in one work item never touch another work item’s files.
- **Checks, then a person.** Open a pull request from the work item, and require CI and a reviewer’s approval in your branch protection. AQ’s Merge is refused while checks are pending or failing or while the review is older than the head commit, so the merge waits on both.

## Readiness checklist for your repo

Run these against the repository you want agents to work in. Each "no" marks the highest level you can safely run on it.

1. **Tests run in one command.** A new machine can clone the repo and run the full suite with one documented command. Time it.
2. **CI runs on every branch** and its result is visible on the pull request.
3. **Main is protected.** With admin access, run `gh api repos/OWNER/REPO/branches/main/protection` (or `gh api repos/OWNER/REPO/rulesets` for rulesets) and confirm required reviews, required status checks and no admin bypass.
4. **Risky paths have owners.** A CODEOWNERS file names people for auth, billing, migrations and infrastructure, and code owner review is required.
5. **Each agent gets its own environment.** No two agents share a checkout, a port or a writable test database.
6. **Secrets are scoped.** Agents get development credentials only; production keys are not reachable from an agent's shell.
7. **Spending has a ceiling.** A monthly limit and a per-run limit exist for every agent account and routine.
8. **Every action is recorded.** You can answer who started a change, which agent and model ran, what commands it ran and what it cost, for any merged pull request.
9. **Rollback is rehearsed.** Count reverts in the last 90 days with `git log --since="90 days ago" --grep="^Revert" --oneline | wc -l`, and confirm a bad deploy can be undone without reading the code.
10. **Tickets are agent-ready.** A sample of recent tickets carries acceptance criteria specific enough to reject a wrong solution.
11. **Someone is on call** for anything the factory ships, with the authority to pause it.

Items 1 to 5 are the floor for level 3. Items 6 to 10 are the floor for level 4. Before level 5, a team needs all eleven, holdout scenarios the agents cannot edit, and evidence gathered over time, such as a month in which human review caught nothing the scenarios missed. That evidence is one necessary step. Consequential or uncertain changes still go to a human reviewer.

## Frequently asked questions

### What is a dark factory in software development?

A dark factory is a software process where coding agents take a specification and produce shipped code with no person writing or reviewing the code. The name comes from manufacturing plants that run with the lights off. In software, people still write the specs and the validation scenarios, and own incidents; the code itself is written and checked by machines.

### Who came up with the five levels of AI coding autonomy?

Dan Shapiro, in his January 23, 2026 blog post "The Five Levels: from Spicy Autocomplete to the Dark Factory." He modeled the ladder on the NHTSA's levels of driving automation and named the top level after FANUC's robot-run factory. Many later articles reuse his numbering with their own labels.

### Is anyone actually running a dark factory for code?

StrongDM has published the most cited claim: a February 2026 writeup by its CTO describing rules that code is neither written nor reviewed by humans, validated by holdout scenarios and test doubles of third-party services. That is the company's own description. Commenters on the Hacker News discussion of the five levels, including Simon Willison, called the approach unproven, and most public examples, Stripe's included, keep a human review gate.

### What level of AI coding autonomy can most teams reach today?

Level 3, a supervised fleet: several agents working in parallel, each in an isolated environment, with a person approving plans and reviewing every diff. Some task types, such as dependency updates or well-specified bug fixes with strong tests, can run at level 4, where people write specs and review samples or risky paths.

### What is a lights-out software factory?

It is another name for a dark factory: a pipeline that turns specs into deployed software without people in the code path. It needs checks that catch what a reviewer would, automatic rollback, hard spend limits, an audit trail, and a person on call for incidents.

### What is the difference between a software factory and a dark factory?

A software factory is any continuous system that turns a task queue into merged code with coding agents, isolated execution, verification and a review gate. A dark factory is the version with the human review gate removed. Most software factories running today keep the gate.

### Can AQ merge or deploy code automatically?

AQ does not provide production deployment; code reaches production through your own CI/CD. AQ adds no automatic merge step, and merge from the Changes panel is refused while checks are pending or failing or the review is stale. Agents use each person's own GitHub access, so branch protection on your repository is what prevents an unapproved merge.

---

About AQ: AQ is where teams put AI agents to work on everything they do: code that lands as a reviewed pull request, bots that run the jobs that repeat, and an assistant that starts each day, all on the agents and subscriptions they already have, like Claude Code and Codex, on private cloud machines of their own or on their own Mac. Start at [aq.dev](https://aq.dev/).