Prompt Engineering Advice From Top AI Startups

Overview

This episode dives deep into the evolving field of prompt engineering, exploring insights from top AI startups and their approaches to building reliable, scalable systems with large language models (LLMs). The hosts discuss practical strategies, emerging patterns, and real-world examples of how companies are leveraging prompts to create impactful AI products.

Notable Quotes

- Meta-prompting is turning out to be a very, very powerful tool that everyone's using now. – Garry, on the transformative potential of meta-prompting.

- The best founders today are maniacally obsessed with the details of their customers' workflows. – Jared, emphasizing the importance of understanding user needs deeply.

- You want the person on the second meeting to say, 'Wow, I've never seen anything like that,' and take my money. – Garry, on the power of rapid iteration and tailored demos in sales.

🧠 The Anatomy of Effective Prompts

- Diana explains ParaHelp's 6-page prompt, which meticulously defines the role of the LLM, outlines tasks, and provides structured output formats.

- Best practices include breaking prompts into markdown-style sections, using XML-like tags for clarity, and incorporating examples to improve reasoning.

- Jared highlights the challenge of balancing general-purpose logic with customer-specific workflows, avoiding the trap of becoming a consulting company.

- The emerging architecture includes system prompts (high-level API definitions), developer prompts (customer-specific logic), and user prompts (end-user interactions).

🔄 Meta-Prompting and Prompt Folding

- Meta-prompting allows prompts to dynamically improve themselves by analyzing failures and generating better versions. Garry describes how this technique is being widely adopted.

- Diana shares how companies use larger models to refine prompts before deploying them in smaller, faster models for production.

- Debugging prompts is critical; Jared discusses adding escape hatches and debug info to help LLMs flag underspecified tasks rather than hallucinating answers.

🚀 Forward-Deployed Engineers and Vertical AI Success

- The forward-deployed engineer model, pioneered by Palantir, is now a key strategy for AI startups. Founders immerse themselves in customer workflows to build tailored solutions.

- Harj and Diana share examples of startups like Gigar ML and Happy Robot, which close seven-figure deals by rapidly iterating on demos and deploying AI agents on-site.

- Vertical AI agents succeed by embedding customer-specific context into prompts, enabling quick turnaround and high-impact results.

🤖 Evaluations as the True Crown Jewel

- Jared emphasizes that evals (evaluation datasets) are more valuable than prompts themselves, as they provide the rationale behind prompt design and enable continuous improvement.

- Founders must deeply understand their users' workflows to create meaningful evals, often requiring in-person interactions to codify nuanced needs.

- Garry likens this process to Kaizen, where those closest to the work are best positioned to refine it, drawing parallels to Japanese manufacturing excellence.

🧩 Model Personalities and Rubric Design

- Different LLMs exhibit distinct personalities. Harj notes that O3 rigidly adheres to rubrics, while Gemini Pro 2.5 applies them flexibly, reasoning through edge cases.

- Rubrics help standardize outputs but must account for exceptions, especially in nuanced tasks like investor scoring or customer support.

- Diana and Harj discuss how context windows and reasoning traces in models like Gemini Pro enable real-time debugging and iterative improvement.