Synthesis: AI Foundations

The Big Picture

You’ve worked through four lessons: how LLMs generate token by token, which ML paradigms exist, why AI outputs are uncertain, and what sets foundation models apart from classical ML.

Individually, these are useful concepts. Together, they form a mental model for AI product management.

Connections

1. From tokens to product design

LLMs think in tokens and probabilities (Lesson 1). This makes them fundamentally non-deterministic (Lesson 3). Same input, different output — that’s not a bug, it’s the architecture.

For you as a PM: Every AI feature needs a design that handles varying outputs. Deterministic UX thinking doesn’t work here.

2. From ML paradigms to foundation models

ML has many paradigms — supervised, unsupervised, reinforcement learning (Lesson 2). Foundation models use self-supervised learning on massive datasets (Lesson 4), making them applicable to many tasks without task-specific training.

For you as a PM: You don’t need to train a separate model for every feature. But you need to know when a specialized model is still the better choice.

3. From task types to product architectures

Classification and generation are different tasks (Lesson 2) that map to discriminative vs. generative models. Most AI products combine both in pipelines — e.g., classify first, then generate.

For you as a PM: Don’t think in terms of “one model.” Think in pipelines. A support bot classifies the request first, then generates the response.

4. From hallucinations to RAG

Hallucinations (Lesson 1) are a structural consequence of probabilistic prediction (Lesson 3). You can’t train them away, but you can reduce them through RAG grounding on foundation models (Lesson 4).

For you as a PM: The most effective tool against hallucinations isn’t a better model — it’s better context through RAG. But RAG is not a silver bullet: outputs still need validation, and retrieval quality depends on your data pipeline.

5. From temperature to API parameters

Temperature and sampling (Lesson 1) are your direct controls for managing the creativity-accuracy tradeoff (Lesson 3). You apply them through foundation model APIs (Lesson 4).

For you as a PM: You don’t need an ML engineer to steer output quality. Temperature, top-p, and system prompts are PM tools.

The Meta-Insight

AI product management is fundamentally about managing uncertainty — in model outputs, in user expectations, in the rapidly evolving model landscape, and in organizational understanding of what AI can and cannot do.

This is what makes it different from traditional product management. You’re not designing for deterministic systems. You’re designing for probabilities.

Your Foundations Checklist

What you should now be able to do:

Explain to stakeholders what an LLM does (and doesn’t do) — Lesson 1
Choose the right ML approach for a product problem — Lesson 2
Design products that handle uncertainty gracefully — Lesson 3
Select and compare foundation models for a use case — Lesson 4
Make the Build vs Buy vs Blend decision — Lesson 4
Set up an evaluation framework — Lesson 3

If any of these feel shaky, go back to the relevant lesson. These foundations carry everything that follows.

Continue with: AI Strategy

You now know HOW AI works. Chapter 2 answers WHEN AI is the right solution — and when it is not.

Self-Assessment

Three scenarios combining multiple concepts from this chapter. Think through your answer before revealing the solution.

Scenario 1: The Unreliable Summary Bot

Your team built an internal bot that summarizes long customer support tickets. The summaries are mostly good, but sometimes the bot invents details that aren’t in the original ticket. A stakeholder demands: “We need a better model — one that doesn’t hallucinate.” How do you respond?

Solution

A “better model” won’t solve this because hallucinations are a structural consequence of probabilistic prediction (Lesson 3) — you can’t train them away. The more effective lever is RAG grounding (Lesson 4): feed the original ticket as context so the model draws from it rather than generating freely. You should also lower the temperature (Lesson 1) to keep outputs closer to the most probable tokens. And the UX must account for varying outputs — for example, by linking back to the original ticket for verification (Lesson 3).

Scenario 2: Classify or Generate?

Your support team receives 500 tickets daily. Currently they’re manually sorted into 12 categories and routed to specialized teams. Your CEO suggests: “Just let ChatGPT answer the tickets directly.” How do you frame the problem?

Solution

This conflates two fundamentally different ML tasks (Lesson 2): classification (assigning a ticket to a category) and generation (composing a response). The right architecture is a pipeline (Lesson 2, Synthesis Connection 3): first a discriminative model for classification, then — only for suitable categories — a generative model for the response. For classification, a specialized smaller model may outperform a foundation model (Lesson 4), while for generation, a foundation model via API with low temperature makes sense (Lesson 1).

Scenario 3: Foundation Model or Custom Training?

You’re building a feature that reviews legal contracts for problematic clauses. Your CTO wants to train a custom model because “accuracy on legal documents must be as high as possible.” A colleague says: “Just use GPT-4 with a good prompt.” How do you decide?

Solution

The decision maps to the Build vs Buy vs Blend framework (Lesson 4). A foundation model with RAG (the Blend approach) is the sensible starting point: use a foundation model via API, grounded with your specific contract database. Custom training only makes sense once you have enough labeled legal data AND can demonstrate that foundation model performance is insufficient — both unlikely at the start. At the same time, the design must account for probabilistic output (Lesson 3): with legal documents, output should never be presented as definitive, but as suggestions for a lawyer to review. An evaluation framework (Lesson 3) involving legal experts is essential here.

Sources: Building on Lessons 1–4. Anthropic Documentation (2025), Chip Huyen “Designing Machine Learning Systems” (O’Reilly, 2022), Shanahan “Talking About Large Language Models” (2023)