Yesterday, we put three people who build software factories in the same room. Dan Lines, co-founder and COO of LinearB, Dex Horthy, co-founder and CEO of HumanLayer, and Aloke Desai, founding engineer and head of product engineering at Warp. If you couldn't attend, the ideas that came out of the hour are below, and the recording is at the end.
We opened by asking the audience how much of their SDLC already runs without a human.
As it turns out, even people attending a workshop on this are still in the early stages of this transformation, with the majority of attendees having only a step or two of their SDLC running without a human.
Throughout the workshop, we dove deep into what it means to run an effective software factory, how to tell whether it’s working, and where to look today for the easiest wins.
Keep score on autonomy, the share of work that clears the factory without a rescue
The phrase the chat kept using afterward was Dan's:
"I think it's about having night vision."
Getting more of the pipeline running without people in it is easy to brag about, but on its own, it says nothing about whether the output is good. "The lights are getting dimmed," Dan said. What matters is whether you can still see what is happening once they are.
Aloke made a version of the same argument from inside Warp, where the team deliberately does not track how much runs in the dark. They track autonomy, meaning the share of work that gets through the factory without a human having to step in to rescue an agent. Warp puts theirs at 80%. That number reflects how well the factory runs, whereas a dark-versus-not-dark score only counts how many people are in the building.
Last quarter's merges are the environment your agents work in this quarter
"Every change that comes out of the factory is compounded into and impacts every future change."
That was Dex, making the structural case for why measurement matters more here than the manufacturing metaphor suggests. A car leaves the plant and stops affecting the next car. Code never leaves. Everything your factory merged last quarter is the environment your agents work in this quarter, so quality problems do not stay where they happened. They raise the cost of every change downstream of them.
That makes the merge event a poor place to stop measuring. Rework, review load, and the cost of the next change all arrive weeks after the PR closes, and a pipeline tuned only for throughput looks healthy the whole time.
Every question the room asked comes down to evidence
The chat ran hot for the full hour and circled the same set of problems:
- How do you measure code quality when quality has always been a matter of human judgment?
- What do automated merges do to a compliance review?
- Does a PR still mean anything when a person and an agent wrote the change together or separately?
- How you keep developers engaged when the job tilts toward reviewing rather than writing.
All of them touch on the same concept: what evidence do you look at once the pipeline moves faster than any one person can watch?
The signals already exist, and they sit in different places. What merged, what it cost, and what you committed to live in separate systems, and none of them answer the question alone. LinearB consolidates them into a single delivery record, which gives the factory an instrument.
In practice, night vision needs:
- Attribution that survives the volume. Every pull request is tied to whoever opened it, person or agent, so ownership is a recorded fact. gitStream in LinearB routes review by codebase area or risk, so a high-risk change reaches a human and a documentation change does not consume one.
- Signal that outlives the merge. LinearB tracks rework, review load, and where review time lands, past the point where the PR closes.
- Cost measured against delivered work. Token spend is an input. LinearB relates it to what shipped, so the cost of delivered work is a number you can take to finance.
How to track quality when judgment stops scaling
No single metric replaces engineering judgment, and none of the ones below tries to. What changes at factory volume is that judgment stops scaling before the pipeline does, so you need proxies that move when quality slips, such as rework rate, how much review a change consumes, and how often work comes back after it merged.
In LinearB's 2026 Software Engineering Benchmarks Report, drawn from 8.1 million+ pull requests across 4,800+ teams, AI-generated pull requests were accepted at 32.7% within 30 days against 84.4% for human-written ones. That gap is a reason to track which of your AI-generated changes land, measured at your own organization against your own baseline.
Watch the full roundtable and get the guide
The roundtable is available on demand, and the guide behind it, on building the context layer for software factories, covers these metrics and how to stand them up on the SDLC you already run.
You’re not turning the lights back on anytime soon. Get night vision.
