# Inference engineering delivers production-grade AI at scale | LinearB Blog

> Baseten’s Philip Kiely outlines why Inference Engineering has emerged as a critical production discipline. Discover how to optimize the AI stack—from GPU specs and engines like vLLM to model routing—to slash latency, prevent failures, and scale secure internal platforms.

_This is a markdown rendering of a live HTML page on linearb.io, generated for AI/LLM consumption — it is not a markdown-only site. To get the full HTML page instead, request this URL with an explicit `Accept: text/html` header (no wildcard, no markdown preference)._


```json
{
  "@context": "https://schema.org",
  "@type": "BreadcrumbList",
  "itemListElement": [
    {
      "@type": "ListItem",
      "position": 1,
      "name": "Home",
      "item": "https://linearb.io/"
    },
    {
      "@type": "ListItem",
      "position": 2,
      "name": "Blog",
      "item": "https://linearb.io/blog"
    },
    {
      "@type": "ListItem",
      "position": 3,
      "name": "Inference engineering delivers production-grade AI at scale",
      "item": "https://linearb.io/blog/inference-engineering-baseten-philip-kiely-production-ai"
    }
  ]
}
```

[Home](https://linearb.io/)

/

[Blog](https://linearb.io/blog)

/

Inference engineering delivers production-grade AI at scale

# Inference engineering delivers production-grade AI at scale

![Photo of Andrew Zigler](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Headshot3_d7231cbda7?_a=BAVMn6ID0)

By [Andrew Zigler](https://linearb.io/blog/inference-engineering-baseten-philip-kiely-production-ai#andrew-zigler)

|

May 14, 2026

![Blog_Inference_engineering_delivers_2400x1256_fc38bbc40a](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Blog_Inference_engineering_delivers_2400x1256_fc38bbc40a?_a=BAVMn6ID0)

[Philip Kiely](https://linearb.io/dev-interrupted/podcast/philip-kiely-baseten-inference-engineering-education-strategy), Head of AI Education at Baseten and author of Inference Engineering, occupies a role that didn't exist a few years ago. His work sits at the intersection of engineering, education, and go-to-market. This convergence reflects how AI-native companies are fundamentally rethinking how they build, ship, and scale intelligent systems. For engineering leaders navigating the transition from AI pilots to mission-critical production deployments, Kiely's perspective offers a roadmap grounded in the realities of delivering reliable AI at scale.

## Inference engineering makes AI systems reliable in production

Inference engineering is not about selecting the best model or wiring together APIs. It is a distinct engineering discipline focused on delivering reliable, performant AI systems in production.

As Kiely points out, there is no single silver bullet for inference. Teams cannot simply grab some GPUs, add an inference engine, throw them together, and expect the system to scale effectively while hitting frontier performance and maintaining three or four nines of uptime.

The stack is broad. It spans model architecture, GPU characteristics, inference engines like vLLM, SGLang, and TensorRT-LLM, orchestration layers, distributed systems design, and production operations. Engineers who excel in one layer, such as SRE or ML research, often lack context across the rest. Kiely wrote his book to address this gap. His goal is to take an engineer with deep expertise in one specific area and expose them to how all the other pieces fit together across the stack.

Despite rapid surface-level churn from new models and updated tooling, the foundational layer has stabilized enough to teach coherently. The primitives from CUDA through PyTorch to inference engines are settling. The opportunity lies not in incremental tuning but in meaningful optimization. When teams publish 20x or 40x performance improvements rather than 2% gains, it signals that the field remains early and open for engineers willing to engage deeply with the stack.

## AI education shapes how developers build AI systems

AI education is more than documentation or tutorials. It is a market-shaping function that introduces developers to emerging concepts early enough to influence how they define problems and evaluate solutions. Kiely notes that his primary mission is to introduce developers to AI concepts in a way that gets them thinking about inference problems and reliable AI-native tools the exact same way his team does.

This approach treats education as a go-to-market lever. Content, demos, and technical teaching become vehicles for building durable mindshare before buyers fully understand their future needs. The OpenAI SDK became an industry standard not because it was technically superior but because it was first. Developers adopted its patterns, and those patterns persisted even when using other providers.

The role increasingly overlaps with engineering. Coding agents enable education teams to ship sophisticated demos, build programmatic SEO, and create custom publishing systems. But the core function remains distribution and discovery, surfacing technical breakthroughs from across the organization and translating them into externally useful narratives.

Internally, [AI enablement](https://linearb.io/resources/hiring-kit-director-of-ai-enablement) follows a progression from basic vocabulary to functional fluency, and eventually to AI-native practice. At Baseten, this means developing engineers with depth in both traditional engineering domains and customer-facing communication. The goal is to turn engineers into effective public communicators of their work, because technical ideas are always best received when they come directly from the source.

## AI-native go-to-market turns education into category leadership

Traditional go-to-market focuses on market capture, moving existing workloads to your platform. AI-native go-to-market is about market development: teaching new categories and workflows before they solidify across the ecosystem. Early educational touchpoints become strategic investments in shaping standards, assumptions, and developer habits.

Developer advocacy bridges engineering, product, and marketing, identifying valuable technical breakthroughs and turning them into narratives that resonate externally. Kiely views his role not as creating technical alpha, but as discovering it within the company and distributing it to the broader market.

Customer-facing engineering strengthens this motion. At Baseten, a forward deployed engineering team reports to the head of engineering, not sales. These engineers deploy into customer accounts, co-engineer solutions, and feed insights directly back into the [product roadmap](https://linearb.io/resources/improve-using-resource-allocation-for-project-forecasting). This structure creates a culture where engineers across infrastructure, model performance, and core product teams regularly interact with customers to ask questions, gather feedback, and unblock tricky issues.

AI-native companies feel infrastructure and deployment pain earlier and more acutely because inference sits directly in the revenue path. When your model goes down, your product goes down. This tight coupling between go-to-market, engineering, and product decisions forces alignment and accelerates learning cycles in ways that traditional SaaS companies rarely experience.

## Internal AI platforms scale secure adoption across the company

Broad AI adoption requires more than training. It requires infrastructure that lets employees act on new capabilities. At Baseten, a centralized [internal platform](https://linearb.io/blog/what-to-look-for-engineering-productivity-platform) allows anyone in sales, operations, and people teams to build and deploy small AI-powered tools securely behind shared authentication and infrastructure controls.

Kiely points out that letting everyone build on their own can spiral out of control quickly, but you also want to avoid heavy procurement cycles for employees just trying to ship a simple web app.

This platform prevents fragmented experimentation while avoiding heavy procurement overhead for lightweight projects. It is a mechanism for making AI adoption company-wide, discoverable, and reusable rather than isolated inside individual teams. Employees can generate internal apps, host them centrally, and share them across the organization within a standardized, secure environment.

Successful enablement links technical capability to organizational design. Skill development only matters when employees have clear pathways and systems that let them act. Internal platforms provide that pathway, turning latent capability into tangible output. The result is a flywheel: more people building, more tools shared, and more learning distributed.

## Model routing and stack optimization cut latency cost and failures

Latency, cost, and reliability are the core production constraints that emerge when AI features move from pilot to mission-critical. Early deployment patterns break at scale, especially when organizations default to expensive frontier models for every request or rely on external APIs with insufficient uptime guarantees.

Model routing is a critical optimization lever. Kiely often questions why organizations route simple requests through the smartest, most expensive models when users are often just performing basic tasks.

Routing simpler requests to cheaper, faster models reduces cost and improves latency without sacrificing quality. But effective routing requires coordinated decisions across infrastructure, engines, parameters, and production architecture. There is no single technical shortcut.

For AI-native companies, this optimization layer is especially urgent. Poor performance directly affects revenue, retention, and product viability. The stakes are higher, and the feedback loops are tighter. Teams that treat inference as a stack and understand how model architecture, GPU specs, inference engines, and distributed systems interact are positioned to [deliver the reliability](https://linearb.io/solutions/predictable-software-delivery) and performance that production demands.

## Treating AI as a foundation, not a feature

Philip Kiely's work at Baseten and his book Inference Engineering reflect a broader shift in how AI-native companies approach engineering, education, and go-to-market. The role of AI education is to inform and to shape how developers think about problems before those problems are fully defined.

The challenge of inference engineering is technical and organizational, requiring depth across a broad stack and close collaboration between engineering, product, and customers. The opportunity for optimization remains vast, signaling that the field is still early and open for leaders willing to engage deeply with the fundamentals.

For engineering leaders, the lesson is clear. AI adoption at scale requires more than tools and training. It requires infrastructure, culture, and strategic alignment across the organization. The companies that succeed will be those that treat AI not as a feature but as a foundation, building systems, teams, and go-to-market motions that reflect that reality.

To dive deeper into the world of inference engineering and AI-native go-to-market, listen to Philip Kiely's full episode on the Dev Interrupted podcast.

## Improve developer productivity with LinearB

Find us on

[](https://www.linkedin.com/company/linearb)
[](https://devinterrupted.substack.com/)

![Headshot3_d7231cbda7](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Headshot3_d7231cbda7?_a=BAVMn6ID0)

## Andrew Zigler

Andrew Zigler is a GTM Engineer at LinearB and the host of Dev Interrupted, a twice-weekly podcast and newsletter where 40k+ builders decode the transition to AI-native development and agentic orchestration. A classicist by training with a degree from The University of Texas at Austin, Andrew spent his early career teaching in Japan before channeling his interdisciplinary instincts into the tech world. His polymath background informs everything he builds, from automated workflows to the stories he tells about the seismic shifts reshaping software creation.

### Connect with

[](https://www.linkedin.com/in/andrewzigler)
[](https://substack.com/@zigler)
[](https://x.com/andrewzigler)

## Your next read

[![Cover image for You can now measure AI ROI with LinearB](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Blog_Post_Name_2400x1256_1_dd1f47bae7?_a=BAVMn6ID0)](https://linearb.io/blog/ai-roi-dashboard)

AI

[You can now measure AI ROI with LinearB](https://linearb.io/blog/ai-roi-dashboard)

Today, we're launching the AI ROI dashboard in LinearB, live now for every customer with metrics builder enabled.

[![Cover image for AI agents are killing the pull request and reinventing CI/CD](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Blog_Post_Name_2400x1256_5_496cdfde2d?_a=BAVMn6ID0)](https://linearb.io/blog/circleci-rob-zuber-ai-agents-pull-request-cicd-sdlc)

AI

[AI agents are killing the pull request and reinventing CI/CD](https://linearb.io/blog/circleci-rob-zuber-ai-agents-pull-request-cicd-sdlc)

CircleCI CTO Rob Zuber explains why AI adoption is killing the pull request and forcing engineering teams to completely reimagine the software development...

[![Cover image for Rippling's employee graph turns HR data into an agentic AI platform](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Blog_Post_Name_2400x1256_be5ec4183f?_a=BAVMn6ID0)](https://linearb.io/blog/rippling-albert-strasheim-employee-graph-agentic-ai)

AI

[Rippling's employee graph turns HR data into an agentic AI platform](https://linearb.io/blog/rippling-albert-strasheim-employee-graph-agentic-ai)

Rippling CTO Albert Strasheim explains why the employee graph serves as the essential context layer for enterprise agentic AI. Discover how unifying HR, IT,...

## Structured data

_Machine-readable metadata (JSON-LD) embedded in the page for search/AI context — not content rendered on the page itself._

```json
{
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "LinearB",
  "url": "https://linearb.io/",
  "logo": "https://assets.linearb.io/image/upload/v1715628027/logo-mark-lg.svg",
  "description": "LinearB is the engineering productivity platform that helps engineering leaders prove AI is improving throughput without sacrificing delivery confidence, flow efficiency, or developer experience.",
  "sameAs": [
    "https://www.linkedin.com/company/linearb"
  ],
  "award": [
    {
      "@type": "Award",
      "name": "LinearB is a Leader in the 2026 Gartner® Magic Quadrant™ for Developer Productivity Insight Platforms",
      "dateAwarded": "2026",
      "awardedBy": {
        "@type": "Organization",
        "name": "Gartner®"
      }
    },
    {
      "@type": "Award",
      "name": "Great Place to Work Certification",
      "dateAwarded": "2025-2027",
      "awardedBy": {
        "@type": "Organization",
        "name": "Great Place to Work"
      }
    },
    {
      "@type": "Award",
      "name": "America's Best Startup Employers 2025",
      "dateAwarded": "2025",
      "awardedBy": {
        "@type": "Organization",
        "name": "Forbes Magazine"
      }
    }
  ],
  "hasCertification": [
    {
      "@type": "Certification",
      "name": "SOC 1 Type 2"
    },
    {
      "@type": "Certification",
      "name": "SOC 2 Type 2"
    },
    {
      "@type": "Certification",
      "name": "GDPR Compliance certification"
    },
    {
      "@type": "Certification",
      "name": "ISO 27001"
    }
  ]
}
```

```json
{
  "@context": "https://schema.org",
  "@type": "Article",
  "headline": "Inference engineering delivers production-grade AI at scale",
  "url": "https://linearb.io/blog/inference-engineering-baseten-philip-kiely-production-ai",
  "author": {
    "@type": "Person",
    "name": "Andrew Zigler"
  },
  "datePublished": "2026-05-14T07:00:00.000Z",
  "dateModified": "2026-05-14T07:00:00.000Z",
  "image": "https://assets.linearb.io/image/upload/v1778883715/Blog_Inference_engineering_delivers_2400x1256_fc38bbc40a.png",
  "publisher": {
    "@type": "Organization",
    "name": "LinearB",
    "logo": "https://assets.linearb.io/image/upload/v1777485755/linearb-logo-2026.png"
  },
  "description": "Baseten’s Philip Kiely outlines why Inference Engineering has emerged as a critical production discipline. Discover how to optimize the AI stack—from GPU specs and engines like vLLM to model routing—to slash latency, prevent failures, and scale secure internal platforms."
}
```

## More on linearb.io

### Top navigation

- [Book a Demo](https://linearb.io/book-a-demo)
- [AI Code Reviews — Catch security risks, bugs, and spec mismatches](https://linearb.io/platform/ai-code-reviews)
- [AI & Productivity Insights — See how AI tools affect cycle time and delivery speed](https://linearb.io/platform/ai-developer-productivity-insights)
- [Measure AI Impact — Track AI adoption and tie it to delivery outcomes](https://linearb.io/use-case/measure-ai-impact)
- [MCP Server — Chat with your data to spot patterns and boost output](https://linearb.io/platform/mcp-server)
- [Resource Allocation — Cost initiatives and shape your investment strategy](https://linearb.io/platform/resource-allocation)
- [Cost Capitalization — Capitalize engineering costs with audit-ready reports](https://linearb.io/platform/cost-capitalization)
- [Dev Team Management — Set targets and tie throughput to business outcomes](https://linearb.io/platform/goals-and-reporting)
- [DevOps Workflow Automation — Policy-based PR routing, approvals, and tests](https://linearb.io/platform/ai-workflow-governance)
- [AI Powered Support — Unify AI and human code delivery in one clear view](https://linearb.io/use-case/ai-powered-support)
- [Optimization — Surface friction with feedback and MCP insights](https://linearb.io/platform/developer-experience)
- [Reporting — Spot what's working and what needs attention](https://linearb.io/use-case/measuring-developer-experience)
- [Surveys — Turn developer feedback into actionable signals](https://linearb.io/platform/developer-surveys)
- [Platform overview](https://linearb.io/platform/overview)
- [Watch now](https://linearb.io/resources/engineering-productivity-gap)
- [Customers](https://linearb.io/customers)
- [Pricing](https://linearb.io/pricing)
- [Why choose LinearB — Explore your data. Measure performance. Act to improve it.](https://linearb.io/why-linearb)
- [APEX framework — The operating model for AI-era engineering teams](https://linearb.io/resources/apex-framework)
- [Anti-FAQ — The questions other vendors won't answer](https://linearb.io/why-linearb/anti-faq)
- [Security — Enterprise-grade compliance and zero code access](https://linearb.io/security)
- [Build vs. buy — The hidden cost of building it yourself](https://linearb.io/resources/build-vs-buy)
- [Dev Interrupted Podcast — Conversations with engineering leaders](https://linearb.io/dev-interrupted/podcasts)
- [Reports & Guides — Deep dives on productivity and delivery](https://linearb.io/resources)
- [Webinars — Expert sessions on productivity and AI](https://linearb.io/resources?category=workshops)
- [Metrics Benchmarks — See how your engineering org stacks up](https://linearb.io/resources/software-engineering-benchmarks-report)
- [Blog — Product updates and practical insights](https://linearb.io/blog)
- [Help Center — Documentation, setup, and support](https://linearb.helpdocs.io)
- [API Docs](https://docs.linearb.io/api-overview)
- [Status](https://www.linearbstatus.com/)
- [Integrations](https://linearb.io/integrations)
- [LinearB Library](https://linearb.io/library)
- [Engineering metrics](https://linearb.io/library/engineering-metrics)
- [Platform engineering](https://linearb.io/library/platform-engineering)
- [Engineering glossary](https://linearb.io/library/engineering-glossary)
- [Developer productivity](https://linearb.io/library/developer-productivity)
- [AI in software development](https://linearb.io/library/ai-in-software-development)
- [Engineering management](https://linearb.io/library/engineering-management)
- [Developer experience](https://linearb.io/library/developer-experience)
- [DevOps](https://linearb.io/library/devops)
- [Engineering operations and the context layer](https://linearb.io/library/engineering-operations)
- [Engineering efficiency](https://linearb.io/library/engineering-efficiency)
- [Software delivery](https://linearb.io/library/software-delivery)
- [Research and data](https://linearb.io/library/engineering-benchmarks-and-research)
- [LinearB is a Leader in the 2026 Gartner® Magic Quadrant™ for Developer Productivity Insight Platforms](https://linearb.io/resources/gartner-magic-quadrant-dpi-platforms-2026)
- [Sign in](https://app.linearb.io/login)
- [Enterprise](https://linearb.io/solutions/enterprise)
- [Contact](https://linearb.io/contact-us)
- [About us](https://linearb.io/about-us)
- [Careers](https://linearb.io/careers)
- [Service agreement](https://linearb.io/services-agreement)
- [Privacy policy](https://linearb.io/privacy-policy)
- [DPA](https://linearb.io/data-processing-agreement)
- [Security FAQ](https://linearb.io/security-faq)
- [Substack](https://devinterrupted.substack.com/)

### Footer

_Additional links from the site footer, not repeated from the top navigation above._

- [GitHub](https://github.com/linear-b)
- [LinkedIn](https://www.linkedin.com/company/linearb)
- [Twitter](https://twitter.com/LinearB_Inc)