When Karthik Ramgopal, Distinguished Engineer at LinkedIn, describes the company's approach to AI transformation, he doesn't lead with flashy demos or productivity metrics. Instead, he frames it as an infrastructure problem, one that requires the same discipline, systems thinking, and long-term planning that defined the previous generation of platform engineering.
It's about building durable primitives that will outlast the current wave of AI vendors and enable continuous adaptation as the technology evolves. It is not about buying the latest tools or chasing hype. For engineering leaders navigating their own AI transformations, Ramgopal's insights offer a pragmatic roadmap grounded in production experience at massive scale.
LinkedIn is scaling AI with durable agentic primitives
LinkedIn's agentic platform operates on two parallel tracks, each serving distinct but overlapping needs. Ramgopal notes that the term "agent platform" is often overloaded. At LinkedIn, it exists in two primary incarnations. The first supports internal productivity across engineering, product, design, legal, marketing, and operations. The second provides the framework for customer-facing production agents serving LinkedIn's members and enterprise customers. While a lot of the technology is shared between the two, the usage is obviously different.
At the architectural core are reusable building blocks like prompt management, inference abstraction, orchestration primitives, and context handling. These started as simple chain-based systems but evolved into more sophisticated graph-based and harness-based implementations as the team's understanding deepened.
The critical insight that shaped this architecture was recognizing that acceleration must be universal, not isolated. Ramgopal uses a vivid metaphor to explain this dynamic. He notes that building a freeway with multiple lanes that suddenly ends at a dirt road does not solve congestion, it simply moves the bottleneck. Similarly, accelerating only the coding function creates frustration when adjacent teams can't keep pace. He points out that many people mistakenly limit acceleration only to coding, but it extends to how teams write design specs and internal documentation.
This philosophy drove LinkedIn to adopt a decentralized operating model where platform teams serve both human and agentic users. Central standards prevent fragmentation while preserving space for experimentation, a balance that requires continuous judgment calls about what to standardize versus what to leave flexible.
The platform investments function as durable primitives that support ongoing build-versus-buy decisions. Rather than repeatedly integrating new tools from scratch, teams can adopt emerging capabilities without reworking core infrastructure. This approach reduces integration debt and accelerates the adoption cycle for genuinely valuable innovations.
LinkedIn makes context infrastructure the key to useful AI
When asked about the fundamental constraint on AI usefulness, Ramgopal's answer is unequivocal. He states that at the end of the day, it all boils down to a giant context management problem. No matter how sophisticated external tools become, they remain limited without access to proprietary data, workflows, systems, and organizational practices.
LinkedIn treats context as a long-lived infrastructure problem rather than a prompt-level tactic. The goal is making tools effective across changing vendors, interfaces, and workflows, a requirement that demands portable, standards-based solutions.
The team's commitment to open standards like the Model Context Protocol (MCP) reflects this long-term thinking. By exposing internal capabilities through standardized interfaces, LinkedIn improves interoperability, procurement flexibility, security review processes, and long-term maintainability. Ramgopal stresses that it is vital to stick to open standards when exposing proprietary context so that as the tools inevitably evolve, LinkedIn can easily keep up and avoid being left behind.
Context delivery must support different access modalities for humans and agents, even when both rely on the same underlying knowledge and system interfaces. A human might need a web dashboard or Slack integration, while an agent consumes structured data through CLI commands, but the foundational context layer serves both.
The path to building this infrastructure started with visible failure cases. Ramgopal's team deliberately "poked the balloon" of AI hype by demonstrating where missing context caused poor outcomes. They then built prototypes showing how context injection improved results, making the case for repeatable abstractions rather than one-off fixes.
This led to a critical organizational shift where platform teams now recognize they have two classes of users. Ramgopal explains that platform teams must now build very different primitives to support human users and agentic users because their access modalities are entirely different.
LinkedIn uses memory to make agents improve over time
Memory extends context management beyond single sessions, creating agents that improve over time rather than starting fresh with each interaction. Ramgopal explains that memory is primarily a way to help with context management across multiple sessions. LinkedIn's approach begins with working memory for conversation history, coupled with retrieval mechanisms that reduce cost and latency by surfacing only relevant prior context.
The team decomposes longer-term memory into three distinct layers, each serving different purposes:
- Procedural memory captures learned ways of working, including the patterns and preferences defining how tasks should be accomplished.
- Episodic memory focuses on task-specific patterns across sessions, identifying recurring episodes that describe specific intents and extracting the preferences associated with those contexts.
- Long-term memory stores durable user preferences that apply regardless of the specific task at hand, acting as the fundamental working style that persists across all interactions.
This memory architecture enables agents to behave more like reliable collaborators than stateless tools. Rather than starting from zero each time, they build on accumulated understanding and avoid repeating known mistakes.
Memory processing happens largely asynchronously through offline and nearline pipelines. This design choice reflects a crucial insight. Extracting durable signals from noisy data is expensive and doesn't need to happen in real-time. Offline processing allows for more sophisticated analysis without impacting user-facing latency.
The signal sources extend far beyond chat logs. Pull request reviews, team discussions, and other artifacts all feed into memory systems. Ramgopal notes that data can be gleaned from anywhere a team discusses their projects or builds consensus, such as Slack channels or Microsoft Teams.
LinkedIn exposes memory as a reusable sub-agent that other agents query using standard protocols. This architectural choice enables sophisticated capabilities like reconciling conflicting information and determining where to look for specific types of context.
LinkedIn uses evals and guardrails to keep AI production ready
Informal acceptance criteria don't work for AI systems. Ramgopal is blunt about this, arguing that "vibes" aren't reality. Relying on surface-level impressions fails to measure real performance in non-deterministic systems. Unless a team has a robust set of evals, they really don't know how well an agent will perform in the real world.
LinkedIn couples agent-friendly integrations with secure and compliant data exposure, building quality validation, security review, and operational trust together rather than treating them as separate concerns. When the team builds integrations that expose internal systems to AI, they simultaneously build the evals that prove those integrations work correctly.
This approach positions leadership responsibility as finding a middle ground between unchecked AI enthusiasm and blanket skepticism. The goal is introducing guardrails that preserve experimentation while protecting production quality, a balance that requires continuous judgment.
The quality bar is deliberately high. Ramgopal insists that AI-assisted output should closely resemble what a skilled practitioner would have produced manually. This standard makes systems fundamentals more important for engineers, not less, because they need a deep understanding to judge correctness and design quality.
Ramgopal emphasizes using AI as a tutor, not just a doer, especially for junior engineers. Learning, validation, citation checking, and self-testing must remain part of the workflow instead of being sacrificed for short-term speed. Engineers should ask AI to explain its reasoning, provide citations for independent verification, and even test their understanding through AI-generated quizzes.
This philosophy extends to a core principle. Engineers should only use AI to accelerate work they already know how to do. If you don't understand how to accomplish a task manually, you can't judge the quality of AI-generated output or write appropriate validation guardrails. Learn first, then accelerate.
LinkedIn turns two-way mentorship into sustained AI adoption
Sustainable productivity gains require deliberate learning structures. Organizations must create space for questioning, experimentation, and shared understanding, not just raw output acceleration. Ramgopal observes that a new paradigm of two-way mentorship has emerged. While senior engineers traditionally mentored junior engineers, senior engineers are now learning entirely new workflows from those same junior engineers.
LinkedIn's approach centers on this two-way mentorship model. Senior engineers provide judgment, production experience, and system design discipline. Newer engineers contribute native fluency with emerging AI workflows and interaction patterns. Both groups have essential knowledge to share.
This requires intentional cultural work. Open learning cultures where people can ask basic questions safely don't emerge by accident. Ramgopal stresses that organizations must deliberately engineer these moments, as they will not happen by chance. Teams need explicit permission to exchange working practices, share discoveries, and admit confusion without judgment.
LinkedIn uses lightweight assistants in team channels to reduce repeated support effort while preserving human oversight. When someone asks a question, a bot indexed on previous conversations can provide initial answers that humans can correct and refine. This reduces the burden of answering common questions while maintaining quality through human validation.
A model for continuous adaptation
Junior engineers face particular vulnerability to skill atrophy if they use AI before understanding the underlying work. Ramgopal frames this as a leadership responsibility, stressing the need to create structured practice and guided learning opportunities that build genuine capability rather than superficial productivity.
The team's AI-native internship pod model illustrates this philosophy in action. Groups of three to five interns work with an experienced tech lead manager to tackle ambitious, open-ended problems rather than constrained tasks with predetermined solutions. Interns lean on each other and their lead to navigate uncertainty together, developing both AI fluency and systems thinking simultaneously. The experienced lead brings production discipline and design judgment while learning new interaction patterns from AI-native team members.
The through-line in Ramgopal's approach is treating AI transformation as an infrastructure problem requiring the same discipline that defined previous platform generations. Success comes from building durable primitives, maintaining high quality standards, and deliberately engineering learning cultures, not from chasing tools or optimizing for short-term productivity gains.
For engineering leaders, the lesson is incredibly clear. Organizations must invest in context management, memory systems, robust evals, and two-way mentorship. These foundational elements will outlast individual tools and vendors, positioning organizations to benefit from AI advances without repeatedly rebuilding core capabilities. The future belongs to teams that build infrastructure for continuous adaptation, not those that optimize for today's specific tools.
To hear more of Karthik Ramgopal's insights on context infrastructure, agent memory, and scaling AI at LinkedIn, listen to his full episode on the Dev Interrupted podcast.
