Logan Kilpatrick on Building with OpenAI APIs
Guest: Logan Kilpatrick — Head of Developer Relations at OpenAI (now at Google AI).
Host: Lenny Rachitsky
Source: Lenny’s Podcast. Recorded early 2024, shortly after OpenAI launched GPTs and the GPT Store.
Overview
Logan Kilpatrick shares a ground-level view from OpenAI’s developer relations team on how developers and companies should think about building on ChatGPT and the OpenAI API. Covers prompt engineering fundamentals, the GPTs launch, OpenAI’s internal culture, and the go-vertical advice for founders building on top of foundation models. Includes a first-person account of the Sam Altman board crisis from someone who was not in San Francisco at the time.
Key ideas
- Context is all you need. The most important thing in prompt engineering is giving the model enough context. Models are trained to answer the question asked — they do not know who you are, what you want, or what your goals are. Garbage in, garbage out.
- Go vertical. OpenAI will not build domain-specific vertical products (AI sales agent, legal AI, etc.). Founders who pick a specific use case and fine-tune or deeply contextualise the model for it are not competing with OpenAI; they are extending it.
- High agency + urgency. The two hiring criteria Logan identifies most at OpenAI: people who hear about a customer problem and are already working on the solution before being asked; people who do not need 50-person consensus to take action.
- Research team kept intentionally small. In a GPU-constrained environment, adding a researcher who is not dramatically up-levelling the group is a net negative: you now share GPUs with that person and everyone’s experiments slow down.
- AI-augmented humans. It is not AI that replaces people — it is humans using AI tools replacing those who are not.
Prompt engineering
“Context is all you need. Context is the only thing that matters.”
Logan’s framing: prompt engineering is a human skill, not a technical one. When you interact with a person you do not know, you give them context to get a useful answer. Models are the same: they have human-level reasoning but zero prior context about you.
Practical advice:
- Give the model context about who you are, what you are trying to do, and what a good answer looks like.
- For questions about specific people, feed in their writing, tweets, or public material (via Browse) — the model will not know niche individuals well enough from training data alone.
- Few-shot examples (“here is a problem; here is a good answer”) steer the model toward a desired output style without fine-tuning.
The “silly” tricks (smiley faces, “take a break and think”) have a real but small effect — the model is trained on human communication, and signals that correlate with positive human exchanges also correlate with more effort in the response.
Building on the API: the go-vertical argument
Logan’s framework for founders building on OpenAI:
| Use case | OpenAI’s posture | Competitive risk |
|---|---|---|
| General assistant / general agent | OpenAI is building this | High — you must be radically better |
| Vertical, domain-specific tool (legal, sales, medical) | OpenAI will not build these | Low — this is the moat |
OpenAI is optimised for general reasoning, general coding, general writing. Vertical products require domain-specific fine-tuning, proprietary data, and deep integration — things OpenAI deliberately does not do. Harvey (legal AI) is cited as an example of a vertical that OpenAI is not competing with.
GPTs product (context: early 2024)
GPTs (launched November 2023) allow anyone to package a custom system prompt, uploaded files, and tool integrations into a shareable ChatGPT persona. Key points from Logan:
- Before GPTs, sharing a customised ChatGPT experience required sharing a conversation link — ugly UX.
- GPTs let non-developers solve their own problems by encoding context into the model upfront.
- Zapier integration allows all 5,000 Zapier connections to be accessible via a single GPT without code.
- Monetisation of the GPT Store was pending at time of recording.
Note: GPTs have since evolved substantially. This section reflects the 2024 state of the product.
OpenAI operating culture
High agency + urgency. Logan’s two hiring criteria. High agency means hearing about a problem and already pushing toward a solution without waiting for direction. Urgency means not needing multi-department consensus before acting.
Intentionally small research team. GPU constraints make researcher headcount a zero-sum game: each additional researcher who is not dramatically up-levelling others reduces everyone’s experiment throughput. This is the opposite of engineering, where adding a strong engineer is almost always net additive.
Slack-heavy culture. Real-time async communication via Slack allows rapid cross-team coordination regardless of location. Logan cites Sam Altman citing Slack as his most-used app.
Board crisis (November 2023). Logan’s personal account: the crisis happened during a scheduled Thanksgiving company break (the first break OpenAI had taken since ChatGPT launched). Most surprising outcome: by Monday morning, the office was laser-focused and back to work. Logan reads this as evidence of how mission-aligned and high-calibre the team is.