# The end of pre-compute: why the future of AI is on-demand | LinearB Blog

> Chalk co-founder Elliot Marx argues that pre-compute fails in real-time AI by causing latency and errors, advocating for on-demand, real-time data pipelines to lower costs (by only computing necessary data) and improve data scientist workflows.

_This is a markdown rendering of a live HTML page on linearb.io, generated for AI/LLM consumption — it is not a markdown-only site. To get the full HTML page instead, request this URL with an explicit `Accept: text/html` header (no wildcard, no markdown preference)._


```json
{
  "@context": "https://schema.org",
  "@type": "BreadcrumbList",
  "itemListElement": [
    {
      "@type": "ListItem",
      "position": 1,
      "name": "Home",
      "item": "https://linearb.io/"
    },
    {
      "@type": "ListItem",
      "position": 2,
      "name": "Blog",
      "item": "https://linearb.io/blog"
    },
    {
      "@type": "ListItem",
      "position": 3,
      "name": "The end of pre-compute: why the future of AI is on-demand",
      "item": "https://linearb.io/blog/end-of-pre-compute-ai-real-time-data-workflows"
    }
  ]
}
```

[Home](https://linearb.io/)

/

[Blog](https://linearb.io/blog)

/

The end of pre-compute: why the future of AI is on-demand

# The end of pre-compute: why the future of AI is on-demand

![Photo of Andrew Zigler](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Headshot3_d7231cbda7?_a=BAVMn6ID0)

By [Andrew Zigler](https://linearb.io/blog/end-of-pre-compute-ai-real-time-data-workflows#andrew-zigler)

|

December 13, 2025

![Blog_Real_time_AI_infrastructure_2400x1256_121eb774bc](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Blog_Real_time_AI_infrastructure_2400x1256_121eb774bc?_a=BAVMn6ID0)

Early in his career, [Elliot Marx](https://linearb.io/dev-interrupted/podcast/hidden-costs-pre-computing-data-chalk-marx) encountered every engineer’s nightmare: a "money bug." A unit mismatch between training data (dollars) and inference logic (cents) led to a catastrophic error where a system began approving every single loan application.

That scar tissue, earned while building risk infrastructure at Affirm and later co-founding Haven Money, shaped his philosophy on AI infrastructure. Today, as co-founder of Chalk, Marx is tackling the structural flaw that causes these errors: the disconnect between how data scientists work and how production systems operate.

The traditional data infrastructure paradigm has centered on pre-computation. Tools like Databricks and Snowflake excel at processing massive datasets in advance, preparing aggregations and transformations before anyone needs them. But as AI models move to the edge and inference happens in real-time, that approach reveals its limitations.

This article explores Marx's insights on why the future of AI infrastructure is real-time, how this shift counterintuitively lowers costs, and the engineering philosophy required to build it.

## The problem: why pre-compute fails in the real world

Marx points out that the traditional paradigm of crunching numbers overnight breaks down when expensive data must be fetched at the exact moment of inference. This is particularly critical in high-stakes industries like fintech. Marx notes that his team now powers roughly a third of the world's debit card transactions, working with top anti-fraud platforms where P99 latencies under 20 milliseconds are non-negotiable.

The technical challenge goes beyond speed; it is about correctness and security. When you pre-compute, you are often forced to move sensitive data, such as social security numbers and transaction histories, out of secure environments to process them. Real-time architecture solves this by deploying directly into customer cloud environments, ensuring data never leaves the compliance perimeter.

## Solving the 'write it twice' problem

The "money bug" Marx experienced was a symptom of a deeper issue in ML engineering known as the "write it twice" problem. Traditionally, a data scientist builds a model in a Python notebook, validates it, and then hands it off to an engineering team to [reimplement in production](https://linearb.io/blog/software-development-metrics-guide) (often in Java or C++). This handoff is where bugs are born.

Chalk’s architecture bridges this gap by adhering to a simple philosophy: meet data scientists where they live, using the languages they already know: SQL and Python.

To achieve this, they convert Python code to SQL-style expressions using abstract syntax tree (AST) analysis, then execute those expressions through a vectorized query engine built on top of Velox, Facebook's open-source query execution kernel. This allows Python code to run orders of magnitude faster than traditional interpretation while maintaining the same logical semantics. The result is a system where data scientists write code once, and it deploys to production without reimplementation. This effectively eliminates the gap where errors like unit mismatches hide.

## The counterintuitive economics of real-time

One of the most surprising benefits of real-time computation is cost savings. While pre-computing everything sounds efficient, Marx explains that it often results in processing massive amounts of data that will never be used. Moving to real-time allows organizations to compute only on the data they actually need.

A recent case study with Whatnot, the largest live streaming marketplace in the US, illustrates this perfectly. Previously, Whatnot computed predictions for all users daily, then discarded about 90% of them because most users never logged in. By shifting to on-demand computation, they [eliminated that waste entirely](https://linearb.io/blog/finding-eliminating-waste-software-development).

The cost equation becomes even more compelling when using third-party data. For anti-fraud platforms, pre-computing all possible combinations of emails, phone numbers, and social security numbers is mathematically impossible because there are "more combinations than the lifetime of the universe." Real-time computation isn't just cheaper; it is the only viable mathematical approach.

## Building with incrementalism

Building infrastructure for AI requires a different development philosophy than building consumer apps. The complexity is high, and the use cases are novel. To navigate this, Marx advocates for "incrementalism" over the "stealth mode" perfectionism often seen in Silicon Valley. He believes the only way to build a great company is to ship early, get the product into a user's hands, and let their feedback dictate whether it's actually good.

This approach contrasts with the "three years in a room" model of companies like Snowflake. Chalk maintains a Slack channel with every customer, creating an overwhelming but invaluable stream of direct feedback. This creates a [culture where engineers are empowered](https://linearb.io/blog/building-healthy-culture-and-sdlc-consistency-at-super) to work on what excites them, as long as it has a "vector of progress" that aligns with the company's direction.

## The future is on-demand

The future of AI infrastructure is not about replacing existing data platforms, but about complementing them with real-time capabilities. As models move to the edge and inference happens at the moment of need, the ability to fetch, transform, and deliver data in milliseconds becomes as critical as the models themselves.

For engineering leaders navigating this shift, the lessons from Elliot Marx are clear: optimize for the data you actually need, eliminate the "write it twice" gap to reduce risk, and [embrace incrementalism](https://linearb.io/blog/continuous-merge-guide-to-merge-standards) to find the right path forward.

To dive deeper into the future of real-time AI infrastructure, listen to Elliot Marx discuss these ideas in depth on the Dev Interrupted podcast. 

## Improve developer productivity with LinearB

Find us on

[](https://www.linkedin.com/company/linearb)
[](https://devinterrupted.substack.com/)

![Headshot3_d7231cbda7](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Headshot3_d7231cbda7?_a=BAVMn6ID0)

## Andrew Zigler

Andrew Zigler is a GTM Engineer at LinearB and the host of Dev Interrupted, a twice-weekly podcast and newsletter where 40k+ builders decode the transition to AI-native development and agentic orchestration. A classicist by training with a degree from The University of Texas at Austin, Andrew spent his early career teaching in Japan before channeling his interdisciplinary instincts into the tech world. His polymath background informs everything he builds, from automated workflows to the stories he tells about the seismic shifts reshaping software creation.

### Connect with

[](https://www.linkedin.com/in/andrewzigler)
[](https://substack.com/@zigler)
[](https://x.com/andrewzigler)

## Your next read

[![Cover image for Rippling's employee graph turns HR data into an agentic AI platform](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Blog_Post_Name_2400x1256_be5ec4183f?_a=BAVMn6ID0)](https://linearb.io/blog/rippling-albert-strasheim-employee-graph-agentic-ai)

AI

[Rippling's employee graph turns HR data into an agentic AI platform](https://linearb.io/blog/rippling-albert-strasheim-employee-graph-agentic-ai)

Rippling CTO Albert Strasheim explains why the employee graph serves as the essential context layer for enterprise agentic AI. Discover how unifying HR, IT,...

[![Cover image for Deterministic guardrails must evolve to control probabilistic AI systems](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Blog_Post_Name_2400x1256_5_f5375661ec?_a=BAVMn6ID0)](https://linearb.io/blog/launchdarkly-cameron-etezadi-deterministic-guardrails-probabilistic-ai-darkfactory)

AI

[Deterministic guardrails must evolve to control probabilistic AI systems](https://linearb.io/blog/launchdarkly-cameron-etezadi-deterministic-guardrails-probabilistic-ai-darkfactory)

LaunchDarkly CTO Cameron Etezadi explains why traditional engineering pipelines are breaking down in a probabilistic, AI-driven world. Discover how to safely...

[![Cover image for Slack turns channels into the context engine for agentic AI](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Blog_Post_Name_2400x1256_077524ba8b?_a=BAVMn6ID0)](https://linearb.io/blog/slack-jaime-delanghe-mcp-agent-context-channels)

AI

[Slack turns channels into the context engine for agentic AI](https://linearb.io/blog/slack-jaime-delanghe-mcp-agent-context-channels)

Slack Chief Product Officer Jaime DeLanghe breaks down how channels serve as the foundational context layer for human-agent collaboration. Learn why Slack is...

## Structured data

_Machine-readable metadata (JSON-LD) embedded in the page for search/AI context — not content rendered on the page itself._

```json
{
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "LinearB",
  "url": "https://linearb.io/",
  "logo": "https://assets.linearb.io/image/upload/v1715628027/logo-mark-lg.svg",
  "description": "LinearB is the engineering productivity platform that helps engineering leaders prove AI is improving throughput without sacrificing delivery confidence, flow efficiency, or developer experience.",
  "sameAs": [
    "https://www.linkedin.com/company/linearb"
  ],
  "award": [
    {
      "@type": "Award",
      "name": "LinearB is a Leader in the 2026 Gartner® Magic Quadrant™ for Developer Productivity Insight Platforms",
      "dateAwarded": "2026",
      "awardedBy": {
        "@type": "Organization",
        "name": "Gartner®"
      }
    },
    {
      "@type": "Award",
      "name": "Great Place to Work Certification",
      "dateAwarded": "2025-2027",
      "awardedBy": {
        "@type": "Organization",
        "name": "Great Place to Work"
      }
    },
    {
      "@type": "Award",
      "name": "America's Best Startup Employers 2025",
      "dateAwarded": "2025",
      "awardedBy": {
        "@type": "Organization",
        "name": "Forbes Magazine"
      }
    }
  ],
  "hasCertification": [
    {
      "@type": "Certification",
      "name": "SOC 1 Type 2"
    },
    {
      "@type": "Certification",
      "name": "SOC 2 Type 2"
    },
    {
      "@type": "Certification",
      "name": "GDPR Compliance certification"
    },
    {
      "@type": "Certification",
      "name": "ISO 27001"
    }
  ]
}
```

```json
{
  "@context": "https://schema.org",
  "@type": "Article",
  "headline": "The end of pre-compute: why the future of AI is on-demand",
  "url": "https://linearb.io/blog/end-of-pre-compute-ai-real-time-data-workflows",
  "author": {
    "@type": "Person",
    "name": "Andrew Zigler"
  },
  "datePublished": "2025-12-13T08:00:00.000Z",
  "dateModified": "2025-12-13T08:00:00.000Z",
  "image": "https://assets.linearb.io/image/upload/v1765690376/Blog_Real_time_AI_infrastructure_2400x1256_121eb774bc.png",
  "publisher": {
    "@type": "Organization",
    "name": "LinearB",
    "logo": "https://assets.linearb.io/image/upload/v1777485755/linearb-logo-2026.png"
  },
  "description": "Chalk co-founder Elliot Marx argues that pre-compute fails in real-time AI by causing latency and errors, advocating for on-demand, real-time data pipelines to lower costs (by only computing necessary data) and improve data scientist workflows."
}
```

## More on linearb.io

### Top navigation

- [Book a Demo](https://linearb.io/book-a-demo)
- [AI Code Reviews — Catch security risks, bugs, and spec mismatches](https://linearb.io/platform/ai-code-reviews)
- [AI & Productivity Insights — See how AI tools affect cycle time and delivery speed](https://linearb.io/platform/ai-developer-productivity-insights)
- [Measure AI Impact — Track AI adoption and tie it to delivery outcomes](https://linearb.io/use-case/measure-ai-impact)
- [MCP Server — Chat with your data to spot patterns and boost output](https://linearb.io/platform/mcp-server)
- [Resource Allocation — Cost initiatives and shape your investment strategy](https://linearb.io/platform/resource-allocation)
- [Cost Capitalization — Capitalize engineering costs with audit-ready reports](https://linearb.io/platform/cost-capitalization)
- [Dev Team Management — Set targets and tie throughput to business outcomes](https://linearb.io/platform/goals-and-reporting)
- [DevOps Workflow Automation — Policy-based PR routing, approvals, and tests](https://linearb.io/platform/ai-workflow-governance)
- [AI Powered Support — Unify AI and human code delivery in one clear view](https://linearb.io/use-case/ai-powered-support)
- [Optimization — Surface friction with feedback and MCP insights](https://linearb.io/platform/developer-experience)
- [Reporting — Spot what's working and what needs attention](https://linearb.io/use-case/measuring-developer-experience)
- [Surveys — Turn developer feedback into actionable signals](https://linearb.io/platform/developer-surveys)
- [Platform overview](https://linearb.io/platform/overview)
- [Register now](https://linearb.io/event/engineering-productivity-gap)
- [Customers](https://linearb.io/customers)
- [Pricing](https://linearb.io/pricing)
- [Why choose LinearB — Explore your data. Measure performance. Act to improve it.](https://linearb.io/why-linearb)
- [APEX framework — The operating model for AI-era engineering teams](https://linearb.io/resources/apex-framework)
- [Anti-FAQ — The questions other vendors won't answer](https://linearb.io/why-linearb/anti-faq)
- [Security — Enterprise-grade compliance and zero code access](https://linearb.io/security)
- [Build vs. buy — The hidden cost of building it yourself](https://linearb.io/resources/build-vs-buy)
- [Dev Interrupted Podcast — Conversations with engineering leaders](https://linearb.io/dev-interrupted/podcasts)
- [Reports & Guides — Deep dives on productivity and delivery](https://linearb.io/resources)
- [Webinars — Expert sessions on productivity and AI](https://linearb.io/resources?category=workshops)
- [Metrics Benchmarks — See how your engineering org stacks up](https://linearb.io/resources/software-engineering-benchmarks-report)
- [Blog — Product updates and practical insights](https://linearb.io/blog)
- [Help Center — Documentation, setup, and support](https://linearb.helpdocs.io)
- [API Docs](https://docs.linearb.io/api-overview)
- [Status](https://www.linearbstatus.com/)
- [Integrations](https://linearb.io/integrations)
- [LinearB Library](https://linearb.io/library)
- [Engineering metrics](https://linearb.io/library/engineering-metrics)
- [Platform engineering](https://linearb.io/library/platform-engineering)
- [Engineering glossary](https://linearb.io/library/engineering-glossary)
- [Developer productivity](https://linearb.io/library/developer-productivity)
- [AI in software development](https://linearb.io/library/ai-in-software-development)
- [Engineering management](https://linearb.io/library/engineering-management)
- [Developer experience](https://linearb.io/library/developer-experience)
- [DevOps](https://linearb.io/library/devops)
- [Engineering operations and the context layer](https://linearb.io/library/engineering-operations)
- [Engineering efficiency](https://linearb.io/library/engineering-efficiency)
- [Software delivery](https://linearb.io/library/software-delivery)
- [Research and data](https://linearb.io/library/engineering-benchmarks-and-research)
- [LinearB is a Leader in the 2026 Gartner® Magic Quadrant™ for Developer Productivity Insight Platforms](https://linearb.io/resources/gartner-magic-quadrant-dpi-platforms-2026)
- [Sign in](https://app.linearb.io/login)
- [Enterprise](https://linearb.io/solutions/enterprise)
- [Contact](https://linearb.io/contact-us)
- [About us](https://linearb.io/about-us)
- [Careers](https://linearb.io/careers)
- [Service agreement](https://linearb.io/services-agreement)
- [Privacy policy](https://linearb.io/privacy-policy)
- [DPA](https://linearb.io/data-processing-agreement)
- [Security FAQ](https://linearb.io/security-faq)
- [Substack](https://devinterrupted.substack.com/)

### Footer

_Additional links from the site footer, not repeated from the top navigation above._

- [GitHub](https://github.com/linear-b)
- [LinkedIn](https://www.linkedin.com/company/linearb)
- [Twitter](https://twitter.com/LinearB_Inc)