In the rush to adopt generative AI, a dangerous trend is emerging. As Andrew McNamara, Director of Applied Machine Learning at Shopify, puts it, "proper evals seems to be kind of falling off a lot of people's radars." Many teams are relying on "vibe testing" - shipping AI features if they seem good enough after a few tries - rather than conducting the rigorous, human-centered evaluation required to create real value.
McNamara brings a unique and seasoned perspective to this challenge. His journey in AI began long before the current hype cycle, building personal assistants at a startup called Maluuba "before Siri came out." Through his career, from that scrappy startup to applied research at Microsoft to leading ML efforts at Shopify, he has cultivated a balanced approach that bridges scientific discipline with practical, human-focused engineering.
This philosophy is now at the heart of Shopify's AI strategy. Instead of chasing technical novelty, his team is focused on a core question: "Did we fulfill the user's goals? Was the merchant happy?" This article explores how Shopify's commitment to rigorous, human-centered evaluation led to the creation of Sidekick, their AI "co-founder," and how this approach is shaping the future of entrepreneurship.
Shopify Sidekick: an AI co-founder for every merchant
Shopify's commitment to human-centered AI is most evident in Sidekick, its AI assistant designed to act as an expert partner for merchants. The vision is to provide "your co-founder or your eCommerce expert that doesn't judge you," a safe space to ask any question and get expert results.
The inspiration for Sidekick came from a powerful insight: merchants with access to e-commerce expertise in their personal networks were dramatically more successful. Sidekick aims to democratize this expertise, providing an AI assistant that understands both the Shopify platform and each merchant's specific business context.
This is a massive challenge given the diversity of Shopify's merchants, who range from hobbyists to multi-million dollar brands. To serve this spectrum, Sidekick was built with a "massive range of skills," from helping design a logo to running complex business analytics. For new entrepreneurs, it provides step-by-step guidance; for established businesses, it offers natural language access to sophisticated insights.
Vibe entrepreneurship: AI's impact on business creation
As tools like Sidekick mature, they are enabling a new phenomenon that could be called "vibe entrepreneurship" - an approach to business creation where technical barriers are radically reduced. "I think it's just so much easier to be a merchant and to start a business these days," McNamara observes, as AI helps with everything from store design to product selection and branding.
However, this new era of accessibility doesn't eliminate the need for human vision. McNamara stresses that the "entrepreneur mindset, that creativity, that judgment is so critical." AI is not a replacement for the entrepreneur, but a powerful partner to help them jumpstart their business and execute their vision with less friction than ever before.
From vibe coding to real value
In an industry captivated by the "vibe" of generative AI, Shopify's methodical, human-centered approach stands out. It serves as a powerful reminder that the true measure of AI's success is not its technical sophistication, but its ability to create tangible value for the people it serves.
By resisting the allure of shortcuts and committing to rigorous evaluation, Andrew McNamara's team has built more than just a feature; they've built a partner. Tools like Sidekick demonstrate a more mature vision for AI - one where technology acts as an accelerator for human creativity and judgment, not a replacement for it.
For leaders navigating the AI hype cycle, Shopify's journey offers a clear lesson: the most durable and impactful AI solutions will come from the teams that are disciplined enough to focus on fulfilling human goals.
Listen to Andrew McNamara’s Dev Interrupted episode:
