Your PRD Template Doesn't Work for AI Features — Here's the One That Does

A PM ships an AI summarization feature. QA signs off — they tested it on ten documents and it worked. Two weeks post-launch, support tickets start piling up: the summaries are sometimes too short, sometimes miss the key number, sometimes hallucinate a detail that isn't in the source. Engineering points back to the PRD: "it says summarize the document, and it summarizes the document." Nobody's wrong, and nobody's right, because the PRD was written for a system that doesn't behave the way this one does.

This is the most common failure mode in AI product work right now, and it isn't a testing problem or an engineering problem. It's a spec problem. The PRD template you've used for the last decade assumes a deterministic system: give it input A, it produces output B, every time. You write acceptance criteria as binary checkboxes because binary checkboxes are how deterministic systems behave. AI features don't behave that way, and if your PRD doesn't say so explicitly, everyone downstream — engineering, QA, support — will quietly default back to deterministic assumptions and get burned by them.

Why the old PRD format breaks

Ask an LLM to summarize the same document twice with the same prompt and you'll get two different summaries. Neither is "wrong" in the way a broken button is wrong. Quality here isn't a pass/fail property of one output — it's a property of the distribution of outputs across many runs, many inputs, many edge cases. A traditional PRD has no vocabulary for that. It says "the system shall summarize documents accurately," which sounds like a requirement but is actually unfalsifiable. Accurate compared to what? Measured how? Wrong how often before it's a bug versus a shrug?

Three assumptions baked into deterministic PRDs quietly stop being true for AI features:

1. "It works or it doesn't." Replace with: it works X% of the time, measured against a specific test set, and here's what happens the other (100-X)% of the time.
2. "The spec fully determines the behavior." Replace with: the spec constrains the behavior; the model still has freedom inside those constraints, and you need to decide how much freedom is acceptable.
3. "Testing before launch catches the bugs." Replace with: testing before launch catches known failure modes; production traffic will surface new ones, so the PRD needs a plan for what happens after ship, not just before.

If you catch yourself writing "the system should always..." in an AI PRD, stop. Always is a deterministic-system word. Ask what percentage you actually mean, and how you'll measure it.

The six sections a deterministic PRD is missing

Keep your existing PRD skeleton — problem statement, users, success metrics, scope, rollout plan. Add these six sections specifically for the AI-driven parts of the feature. They're the difference between a PRD that reads well and one that actually prevents the "why didn't we catch this" conversation.

1. Model behavior specification

Describe what the model is supposed to do in terms of inputs, constraints, and desired properties — not a literal script of outputs. For the summarization example:

• Input: a document up to 20 pages, plain text or PDF-extracted text.
• Desired properties: preserves the top 3 factual claims from the source; length is 5–10% of source length; no invented entities or numbers not present in source.
• Explicit non-goals: does not need to preserve document formatting; does not need to summarize tables (flag and skip instead).

Notice this isn't "the summary should be good" — it's a checklist an engineer or eval script can actually test against.

2. Acceptance thresholds

Every "must" from a deterministic PRD becomes a "should, at X%, measured by Y." Anchor the threshold to something real, not a round number pulled from thin air:

"The model should preserve all top-3 factual claims in 95% of summaries, verified against a 150-document labeled test set. Below 90%, this feature does not ship."

Where does the number come from? Start by measuring the human baseline the feature is replacing. If your support team currently summarizes tickets by hand with 88% accuracy in 6 minutes, your AI needs to beat that on at least one axis — speed or accuracy — without falling meaningfully below the other. "90% accuracy, matching human baseline" is a defensible number. "99.9% accuracy" invented in a planning meeting is not, and it will quietly get ignored by engineering the first time it's inconvenient.

3. Failure mode catalog

List the specific ways this feature can fail, ranked by how bad each one is. This is the section most PRDs skip, and it's the one support and legal will thank you for later.

Failure mode	Severity	Example	Mitigation
Hallucinated fact	High	Summary states a dollar figure not in the source	Post-generation fact-check against source spans
Missed key point	Medium	Summary omits the deadline mentioned in the doc	Require inclusion of dates/deadlines via structured extraction
Tone mismatch	Low	Summary reads overly casual for a legal document	Prompt-level tone constraint, spot-checked in eval set
Refusal on valid input	Medium	Model declines to summarize a long but benign document	Retry with chunking; alert if refusal rate > 2%

Severity isn't about how often it happens — it's about what it costs when it does. A hallucinated legal fact at 1% frequency is worse than a tone mismatch at 20% frequency. Rank accordingly, and make clear which failure modes block launch versus which are acceptable-with-monitoring.

4. Behavioral constraints (guardrails)

What the model must never do, independent of how well it performs on the happy path. These are your hard boundaries, usually enforced outside the model itself (system prompts, output filters, business logic):

• Never summarize documents marked confidential without an explicit user opt-in.
• Never output PII that wasn't already present in the visible portion of the source document.
• Always include a "generated by AI, verify before relying on it" disclosure for outputs shown to end customers.

These read like the "must" statements from a deterministic PRD, and that's fine — some things really are binary. The skill is telling them apart from the probabilistic requirements above.

5. Evaluation plan

How will you know, ongoing, whether the acceptance thresholds in section 2 are still being met? This section names owners and cadence, not just tools:

• Pre-launch: golden test set of 150 documents, labeled by two reviewers, eval script run on every model/prompt change.
• Post-launch: 1% of production summaries sampled weekly for human review; automated fact-check flags routed to a Slack channel for the on-call PM.
• Trigger for re-eval: any prompt change, model version bump, or a 2-week rolling drop below the 90% floor from section 2.

If your PRD doesn't name who looks at the eval dashboard and how often, the eval dashboard will exist and nobody will look at it.

6. Data and privacy requirements

AI features usually touch more data surface than the traditional feature they're replacing — training data, retrieval context, logged prompts and completions for debugging. Spell out: what's logged, for how long, who can see the raw completions (support? engineering? no one?), and whether any of this data trains future models. This is the section legal and security will actually read, so don't leave it as a TODO.

A worked comparison

Here's the same requirement, written the old way and the new way:

Deterministic-style (wrong)	AI-native (right)
"The system shall accurately summarize uploaded documents."	"Summaries preserve the top 3 factual claims from source in ≥95% of cases (150-doc labeled eval set); below 90%, feature is blocked from launch."
"The chatbot shall never give incorrect information."	"The chatbot refuses out-of-scope medical/legal/financial questions in ≥99% of cases (200-prompt adversarial red-team set), and answers in-scope questions with fact-check-verified accuracy ≥92%."
"Search results shall be relevant."	"Top-3 search results contain a document judged relevant by human raters in ≥85% of a 300-query eval set, benchmarked against current keyword search's 71% baseline."

The pattern: name the metric, name the test set, name the threshold, name what happens if you miss it.

Common pitfalls

• Over-specifying to feel safe. Trying to script every possible AI response ("if the user says X, respond exactly with Y") defeats the point of using a probabilistic system in the first place, and you'll spend the sprint fighting the model instead of shipping. Specify properties and constraints, not scripts.
• Setting thresholds with no baseline. A number with no comparison point ("99% accuracy") is a number nobody can argue with or act on. Anchor every threshold to a human baseline, a competitor benchmark, or a cost-of-error calculation.
• Treating the PRD as frozen at launch. AI products drift — model providers ship silent updates, usage patterns shift, new failure modes appear that your pre-launch eval set never covered. Build a review cadence into the PRD itself, not as an afterthought.
• Skipping the failure catalog because "it mostly works." The failure catalog isn't there because the feature doesn't work — it's there because any AI feature fails some percentage of the time, and deciding which failures are acceptable is a product decision, not something to leave to whoever notices the bug report first.
• Writing the eval plan without naming an owner. An eval plan with no named owner and no cadence is a wish, not a plan.

Next steps

Take your next AI feature PRD and run it through this checklist: does it have acceptance thresholds tied to a measurable test set? A ranked failure mode catalog? A named eval owner? If two or more are missing, that's where the next production incident is going to come from.

If you're a PM working on AI features and want a structured way to build this muscle — writing specs, running evals, and thinking about AI product tradeoffs with people doing the same work — Joinloop's PM cohort covers exactly this: how to scope, spec, and ship AI-native products without pretending they behave like the deterministic ones you're used to.