In the world of AI, there's a massive gap between what works in a research lab and what succeeds in a production environment. Bridging that gap is one of the most critical challenges for modern engineering leaders. Elizabeth Lingg, Director of Applied Research at Contextual AI, has built her career on this very challenge, navigating both worlds at companies like Microsoft and Apple.
Her journey reflects the industry's own evolution. "When I first joined, you either picked, you're an AI researcher or you're a software engineer," she notes. Today, those roles have converged, and practitioners need a full-stack understanding of the entire pipeline. For Lingg, this meant learning to balance the theoretical with the practical. "Once I joined the real world, I have to build these products, I have to make sure the customers are happy, then I start worrying about things like latency," she explains.
This article explores Lingg's disciplined methodology for building specialized AI that delivers real-world impact, from her dual-loop approach to measurement to her focus on "groundedness" and collaborative team structures.
Measuring real-world AI impact with a dual-loop approach
How do you measure success in applied AI? Lingg emphasizes the importance of a dual-loop approach. The first, or "inner loop," involves standard technical metrics like accuracy, precision, and recall. However, these only tell part of the story.
The true measure of success comes from the "outer loop"—metrics that track customer utilization, satisfaction, and engagement. The key is to find the correlation between them. "Can you apply a linear regression and do you see that there's a correlation there?" she asks. "What features should be weighted higher or lower in order to determine how the inner loop influences the outer loop?"
Beyond these quantitative measures, Lingg also stresses the importance of a qualitative assessment, or what she calls the "vibe check." This more intuitive evaluation asks whether the AI solution simply feels right to users, providing a holistic view of its effectiveness.
Ensuring accuracy in specialized AI through groundedness
For many enterprise AI applications, factual accuracy is non-negotiable. Lingg emphasizes that specialized intelligence must be "grounded in the retrieve knowledge," ensuring there are no hallucinations based on the model's general training data.
This is especially critical for domain-specific applications in areas like technical documentation, legal compliance, or specialized code generation, where a general-purpose model's plausible-sounding answer may be dangerously incorrect. To ensure this groundedness, her team at Contextual AI employs specific testing frameworks to verify that AI responses meet strict criteria for accuracy and relevance.
This challenge also extends to human preference alignment. Simply optimizing for what users want to hear can lead to "sycophancy," where a model prioritizes pleasing the user over being accurate. Balancing user preferences with factual accuracy is a complex research area that requires careful, disciplined evaluation.
Building cross-functional teams to bridge research and production
Creating effective applied research teams requires bridging different skill sets. Lingg has found success by encouraging team members to step outside their comfort zones and experience other parts of the development process, such as having researchers observe engineers' workflows. This cross-functional experience builds mutual understanding and fosters a growth mindset.
However, this collaboration needs to be balanced. While it's valuable for team members to expand their horizons, they must still focus on their primary responsibilities. For leaders looking to foster innovation, Lingg recommends starting with small, focused experiments that demonstrate clear value. By creating a culture where specialists can learn from each other while maintaining their core expertise, teams can successfully bridge the gap between research and production.
From theory to impact
The journey from a promising AI model in a lab to a valuable product in the hands of customers is fraught with challenges. As Elizabeth Lingg’s experience demonstrates, success requires a disciplined and holistic methodology that bridges the worlds of research and applied engineering.
This approach balances quantitative "inner loop" metrics with real-world "outer loop" impact, ensuring that technical improvements translate into genuine customer value. It prioritizes groundedness to build trust and reliability in specialized domains. And it is all powered by a collaborative team culture where diverse experts learn from each other.
For leaders in the rapidly evolving field of AI, this balanced approach provides a clear framework for moving beyond theoretical potential to create specialized AI applications that deliver real, measurable impact.
To hear more from Elizabeth Lingg on bridging AI research with real-world impact, listen to her discuss these ideas in depth on the Dev Interrupted podcast.
