Few people have watched the data warehouse mature from both ends of the scale question as closely as Jordan Tigani, co-founder and CEO of MotherDuck. As a founding engineer on Google BigQuery, he helped build one of the largest distributed data warehouses ever shipped, then spent years noticing where its assumptions stopped matching reality. That vantage point became a company built on a very different bet about what data tools should look like.
That bet now sits at the center of a much larger shift. The practitioners of the data stack are increasingly AI agents, and the tools they reach for, along with the humans who orchestrate them, have to change to meet that challenge. In a recent Dev Interrupted conversation, Tigani laid out what that transition looks like. He covered why most workloads never needed a distributed system, how pipelines and dashboards are becoming prompts, and what the data engineer's job turns into when agents do the building.
The data stack was built for machines that no longer exist
The founding insight came from using BigQuery to study BigQuery. When Tigani analyzed how much data customers held and which queries they actually ran, the result was a surprise. Roughly 90% of queries touched about 100 megabytes, a volume so small that nobody inside Google would think twice about it. Even giant customers like HSBC, Home Depot, and Walmart rarely ran anything that big.
Internally, the energy pointed the other way, toward making queries over tens and hundreds of terabytes run faster. The complaints coming back from customers were different. A 100 megabyte query was no faster than a 100 gigabyte one, and there was almost always a human waiting on the result. Adding a second of distributed overhead to every query made the common experience worse in order to serve a rare one, which customers ran infrequently anyway because it was expensive.
That observation lingered for years. Much of today's data infrastructure was designed when machines were small, when 100 gigabytes could not fit in memory and 100 cores meant stitching together a fleet. Basic machines on EC2 now carry hundreds of cores and terabytes of RAM. "If you were gonna build these things now, you would do them differently," Tigani says. Data itself had also grown up, moving out of the specialist's domain and into more tools and more hands. That created demand for smaller, faster, and more personal ways of working with it.
AI agents break the modern data stack's equilibrium
Tigani borrows a framing from a former college professor, who described evolution as punctuated equilibrium, meaning long stretches of stability broken by rapid transitions to a new state. Software tends to move the same way, and the modern data stack was one of those equilibria. Ingest, transform, analyze, and visualize each had its own swim lane and its own companies, and everybody was comfortable. Then agents arrived, and that arrangement started coming apart.
Predicting the exact shape of the change is a recipe for spectacularly bad predictions, but the signals are already visible. Text-to-SQL works, just not in the way anyone expected, and with an agent in the loop it produces high-fidelity queries. Agents like Claude already make very good visualizations, and they will only get better. Run a query, visualize the result, and much of what a BI tool does has collapsed into a prompt.
The collapse reaches into architecture as well as workflow. Distributed warehouses were shaped for a world with humans in the loop, running one query at a time. A fleet of agents working in parallel is a different problem, and systems built for the old pattern fit it poorly.
Pipelines and dashboards become code that agents write
The same logic runs upstream. Bringing data in and transforming it, the unglamorous plumbing of the stack, also turns out to be "highly vibe codable," as Tigani puts it. Point an agent at a source and ask it to pull the data, and it can figure out how, and it keeps getting better at doing so. What used to be a project becomes a request.
MotherDuck leans into that shift with two primitives. Flights are Python scripts that run on a schedule, backed by guides and templates that make it easy for an LLM to write them. Dives are TypeScript visualizations an LLM can generate directly. The key design choice is that pipelines and visualizations emerge from a goal rather than being built up front. One recent morning, Tigani's work started with a new intensity metric, not a pipeline. The computation was heavy, so a flight got created, a backfill ran, and an hourly refresh was scheduled, all pulled into being by the actual objective. The pipeline was never the point, only the thing required to reach it.
Treating that output as code is what keeps it governable. Flights and dives are ordinary source files, so MotherDuck runs them through a GitHub repo where PRs get reviewed and a commit hook syncs them into the product. GitHub becomes the source of truth, so the same gates that discipline engineering work, such as review, cloning, and tweaking, apply cleanly to generated pipelines. When agents decide what SQL to run and push it into a schedule, the check has to live somewhere. Tigani is unequivocal on the point. "These things have to be code and treated like code."
Building in that shape has a compounding benefit, too. As models get better at writing SQL, Python, and visualizations, the features built on top of them improve without a rewrite, rather than getting eaten as the model closes the gap.
Single-node warehouses fit the way agents work
The architectural case sharpens once agents enter the picture. Distributed systems add latency and complexity that the overwhelming majority of queries never needed, and they are slow to evolve. Some BigQuery optimizations took months of work for something relatively simple. On a single-node system, comparable changes land in a week or a weekend.
Modern hardware makes the single node viable again. With hundreds of cores and terabytes of RAM in one machine, most work no longer requires combining machines at all. Skipping the distributed layer buys lower latency and far simpler iteration. That simplicity matters more, not less, as the consumer of the data changes from a person to a swarm.
The fit is almost geometric. A giant shared warehouse is an awkward home for many agents working on different things at once. Tigani instead pictures one warehouse per agent, a single-node system replicated for as many agents as a team runs. Isolating each agent's workload in its own sandboxed space addresses the real fear of letting a fleet loose on shared data. It also maps the infrastructure onto the emerging pattern of work rather than fighting it.
Data engineers become managers of agent fleets
If pipelines, transformations, and visualizations all come from prompts, the human job changes. MotherDuck has been sketching what that looks like in a framework called Watertown, a tongue-in-cheek nod to Steve Yegge's Gas Town, which imagines the roles, queues, and escalation paths for coordinating agents on data work. Whether every specific role is needed is an open question. Any team, though, will need a way to hold a queue of tasks and a mechanism to surface the important ones to humans.
The center of gravity moves from building to reacting. "A data engineer is really gonna be sort of a manager of a fleet of agents that are gonna be doing data work," Tigani predicts, and much of that management is handling what breaks. Data engineering is uniquely fragile here. Unlike software, it can break through no fault of the person who built it, because the inputs themselves change. Schema drift, a stalled or buggy upstream source, or a shift in the distribution or format of the data can all invalidate a perfectly correct pipeline. Some of that repair can be automated too. If a field gets renamed or a column switches from whole numbers to decimals, an agent can often chase the change through every downstream visualization.
The payoff extends past the engineers. MotherDuck calls them NTDs, or non-technical ducks, meaning the people who once assumed they lacked the background to interrogate their own sales book or marketing campaigns. Now they can ask business questions directly, which is genuinely liberating, even as the new barrier becomes knowing what to ask. Meanwhile, specialists move up a level, overseeing and correcting agent output at scale rather than hand-writing every query. Even the person who once wrote the most SQL at the company can't remember the last time he wrote any by hand.
The next equilibrium rewards teams willing to simplify
None of this has settled into a tidy end state. Token usage is rocketing upward, CFOs are in the room, and companies are still discovering which levers constrain spend and when pulling them makes sense. A great deal will break before the next equilibrium arrives.
The through line still holds. Expectations of what technology can do shift faster than anyone plans for. From BigQuery to MotherDuck to agents, the winning move has repeatedly been the willingness to take things away, simplify, and trust a new definition of good. What endures is a fluency layer between humans and agents. It leaves shareable artifacts instead of one-off answers, so the gains of any single breakthrough can be spread around rather than reinvented ten times over.
Listen to Jordan Tigani on the Dev Interrupted podcast to hear more about rebuilding the data stack for a post-scale world, single-node versus distributed warehouses, and data engineers managing fleets of AI agents.
