Understanding how developers interact with your APIs is crucial for building successful products. Sagar Batchu, co-founder and CEO of Speakeasy, emphasizes that tracking the right metrics can transform how organizations approach API development and maintenance.

"Caring about your APIs, the way that it manifests is understanding customer usage," explains Batchu. "You probably wanna track which parts of the API are getting used the most versus the least, like error rates. There's a great metric I like to call time to 200, which is the time to your first successful call as a user."

This "time to 200" metric has emerged as a north star for many API-first companies. It measures how quickly a new developer can go from discovering your API to making their first successful call, a critical indicator of your API's usability and documentation quality. Over time, this metric should trend downward as you refine your developer experience.

Beyond this key performance indicator, Batchu recommends monitoring which endpoints receive the most traffic versus those rarely used. This usage pattern analysis can inform future development priorities and help identify parts of your API that might be unnecessarily complex or poorly documented.

Error rates provide another valuable window into developer experience challenges. When certain endpoints consistently generate errors, it signals potential issues with documentation, parameter requirements, or the underlying implementation that need addressing.

For engineering leaders, these metrics offer actionable insights that can drive continuous improvement in API design and implementation. By establishing baseline measurements and tracking progress over time, teams can quantitatively demonstrate the impact of their developer experience investments.

LLMs and APIs Are A New Class of Consumers

The rise of Large Language Models (LLMs) has introduced a new type of API consumer: one that interacts with APIs at unprecedented scale and frequency. This shift is creating both challenges and opportunities for API providers.

"As these new interfaces come online, there's even more of a demand to actually do the traditional things well," Batchu points out. "Even more because there's just more API consumers now and there's more API calls and there's gonna be more usage."

One of the most striking observations is the sheer volume of API calls generated by LLM integrations. According to Batchu, LLMs typically generate "probably doing 10 times as many API calls as a standard static integration would do." This order-of-magnitude increase requires API providers to ensure their infrastructure can handle the load while maintaining performance.

Interestingly, LLMs benefit from many of the same developer experience elements that human developers need. Clear documentation, examples, and consistent design remain essential—perhaps even more so—when your API consumers include both humans and AI systems.

"The simpler API is the better documented described examples, like it's all compounding in value," notes Batchu, highlighting that well-structured APIs perform better with LLMs than complex, poorly documented ones.

Context is particularly critical when designing APIs for LLM consumption. When LLMs have access to comprehensive documentation, examples, and schemas, they can make more intelligent decisions about how to use your API effectively. Without this context, their ability to leverage your API's capabilities is significantly diminished.

For engineering leaders, this means that investments in API documentation, examples, and clear design patterns now serve a dual purpose: improving both human developer experience and enabling effective LLM integration.

Model Context Protocol (MCP) is Transforming LLMs into Operational Tools

The Model Context Protocol (MCP) represents a significant advancement in how LLMs interact with APIs, transforming them from discovery tools into operational tools with deterministic outputs.

"With MCP in particular, it basically is a protocol, a schema that defines the resources of an API an agent has access to, or an LLM has access to when calling that API," explains Batchu. This protocol provides the structure and context necessary for LLMs to make intelligent decisions about when and how to use API tools.

What makes MCP particularly powerful is that it enables automatic decisioning by the LLM. "The cool thing is they're not even prompted. They're not even saying, please use this tool. They just start, you just continue with what you wanted to do," Batchu notes. This seamless integration allows users to work naturally with the LLM while it determines when API calls are needed to fulfill the request.

For API providers, implementing MCP is relatively straightforward, especially with tools like Speakeasy that can generate MCP servers directly from API specifications. The implementation effort is minimal compared to the potential benefits of making your API accessible to the growing ecosystem of LLM-powered applications.

Batchu recommends starting with internal workflows when exploring MCP implementation: "I would actually start by, even before investing in the stuff for your product, investing for your productivity of your team." This approach allows organizations to gain experience with the technology while delivering immediate value through improved internal processes.

By enabling LLMs to interact with APIs in a structured, deterministic way, MCP bridges the gap between the flexibility of natural language interfaces and the precision required for operational tasks. For engineering leaders, this represents an opportunity to make their APIs more accessible while maintaining control over how they're used.

Agent Experience (AX) is The Evolution of Developer Experience

As LLMs and agentic AI systems become more prevalent API consumers, a new concept is emerging in the industry: Agent Experience (AX), an evolution of traditional Developer Experience (DevEx).

"The industry's already kind of gravitating to terms like agent experience AX instead of DevEx," Batchu observes. "It does come down to actually a lot of the same things."

This shift recognizes that while the consumers may be different, many of the fundamental principles remain the same. Clear documentation, examples, and consistent design are still essential—perhaps even more so when your API needs to be understood by both humans and AI systems.

However, there are important nuances to consider when designing for agent experience. Context becomes even more critical, as agents rely heavily on the information provided to make decisions about how to use your API. Simplicity also takes on new importance.

"I always tell 'em to start small and with agents, I think it's even more true," advises Batchu. "Like context is really important." Rather than exposing your entire product functionality through an API, focus on the core operations that deliver the most value. This targeted approach makes it easier for agents to understand and effectively use your API.

For engineering leaders, embracing agent experience doesn't mean abandoning developer experience principles. Instead, it means extending those principles to accommodate a new class of consumers with their own unique needs and behaviors. By doing so, organizations can position their APIs to thrive in an increasingly agent-driven ecosystem.

Organizational API Strategy

Creating exceptional APIs isn't just a technical challenge, it's fundamentally an organizational one. As companies scale, maintaining consistency across multiple APIs becomes increasingly difficult without the right governance structure.

"Creating a center of excellence for API design is crucial because it shifts the focus from just being a technical challenge to an organizational one," Batchu explains. This center of excellence approach ensures that someone within the organization has ownership over how APIs look and feel, establishing and maintaining standards across the board.

For smaller companies, Batchu recommends having a single owner for APIs: "The most successful API strategies I've seen start with a single owner who ensures quality and standards across the board." This approach simplifies governance and ensures consistency when the number of APIs is limited.

As organizations grow, however, Conway's Law comes into play: organizations tend to produce APIs that mirror their internal structure. "If you have five teams, your API is gonna look like it was designed by five different people," Batchu notes. To counter this tendency, larger organizations need platform teams focused on governance and standards.

"As your company scales, maintaining a consistent design experience across all APIs becomes critical," emphasizes Batchu. This consistency is achieved through what he calls a "last mile gate," a governance process that ensures all APIs adhere to organizational standards before being released.

Interestingly, many of the API adoption challenges organizations face are related to internal discovery and cataloging rather than external API design excellence. "I think for companies focusing on this as an internal problem, I would actually start by saying a lot of the first step really comes down to like discovery and cataloging and making sure you have the tools in place to know all the APIs that you have," Batchu advises.

For engineering leaders, this means investing in both the technical aspects of API design and the organizational structures needed to govern APIs at scale. By doing so, they can ensure that their APIs present a consistent, high-quality experience regardless of which internal team developed them.

Excellence in the API Economy

As we've explored throughout this article, building successful APIs requires attention to multiple dimensions: measuring developer productivity, adapting to new consumers like LLMs, implementing emerging protocols like MCP, evolving from developer experience to agent experience, and establishing effective organizational governance.

What unites all these dimensions is a commitment to excellence—a recognition that APIs are not just technical interfaces but critical business assets that deserve investment and care. As Batchu points out, "APIs are more an organizational problem than a technical one," requiring thoughtful leadership and strategic vision.

For engineering leaders navigating this landscape, the key takeaways are clear: measure what matters, start small and focused, invest in documentation and examples, embrace new protocols like MCP, and establish clear ownership and governance structures. By doing so, organizations can build APIs that deliver value to both human developers and AI agents, driving adoption and business success in an increasingly API-driven world.

As we look to the future, the fundamentals of API excellence will remain important even as the technologies and consumers evolve. By focusing on these fundamentals while embracing innovation, engineering leaders can ensure their APIs remain relevant, usable, and valuable in the rapidly changing technology landscape.

Listen to Sagar Batchu's full Dev Interrupted episode below, and read the full transcript here.