Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor)

Overview

This episode explores the meteoric rise of Mercor, the fastest-growing company in history, and its role in the AI ecosystem. Brendan Foody, CEO and co-founder, explains how Mercor helps AI labs by hiring experts to create evaluations (evals) that measure and improve AI model capabilities. The conversation delves into why evals are critical for AI progress, the future of work in an AI-driven economy, and the values and strategies that fueled Mercor’s unprecedented growth.

Notable Quotes

- If the model is the product, then the eval is the product requirement document. — Brendan Foody, on the importance of evals in AI development.

- The wealthiest companies in the world are willing to spend whatever it takes to improve model capabilities. — Brendan Foody, on the demand for AI advancements.

- AI won't replace you. People that are really good with AI will replace you. — Lenny Rachitsky, summarizing the importance of leveraging AI tools.

🧠 The Era of Evals

- Evals are becoming the critical bottleneck for AI progress, serving as both the product requirement document and sales collateral for AI models.

- Labs use evals to measure success, train models, and demonstrate capabilities. For example, a lawyer might create a rubric to evaluate a model’s ability to redline contracts.

- The shift from crowdsourced, low-skill data to high-caliber professionals (e.g., software engineers, doctors, lawyers) is transforming the AI training landscape.

🚀 Mercor’s Unprecedented Growth

- Mercor scaled from $1M to $500M in revenue in just 17 months, making it the fastest-growing company in history.

- The company’s success stems from identifying a massive market need: sourcing experts to help AI labs improve model capabilities.

- Brendan credits Mercor’s growth to focusing on leading indicators in fast-moving markets, customer obsession, and maintaining high standards.

💼 The Future of Work in an AI Economy

- Brendan challenges the narrative of job displacement, emphasizing the creation of new roles like AI eval experts.

- Jobs with elastic demand, such as software development and product management, will thrive as AI increases productivity.

- Skills that leverage AI tools will become essential. Brendan advises embracing AI to amplify capabilities rather than resisting its adoption.

🌍 Building a Global Labor Marketplace

- Mercor envisions a unified global labor market, where AI automates the matching of talent to opportunities.

- The company’s platform pays experts a median rate of $95/hour, with top professionals earning up to $500/hour.

- Brendan highlights the importance of creating dignified, high-quality experiences for workers, contrasting Mercor’s approach with traditional crowdsourcing platforms.

🏗️ Core Values Driving Success

- Can-Do Attitude: Setting ambitious goals and aligning the company’s trajectory to achieve them.

- High Standards: Hiring exceptional talent, including former founders and industry leaders.

- Intensity: Cultivating a high-output culture focused on delivering results.