Everyone repeats this sentence that doesn’t reflect the reality (the eval is the new PRD). That’s going to be perceived as one of 2 things
stop writing PRDs, really?
or "let engineers write more tests. doable?
The PRD existed because building was really expensive and with what’s going around us, it’s not anymore. Think of a prototype during your afternoon, by morning you can have it live to test it. So describing what to build stopped being the valuable document.
So, what is valuable ? Actually, describing what good looks like, written so it can be checked all the time is now the job. And that’s the simple definition of eval and where your decision lives.
Ask yourself why evals matter? you can think of it that —> a demo is a sample of three, picked by its biggest fan. It proves the thing can work, never how often or for whom. The faster you build, the more convincing your bad ideas look. So the PM becomes the filter engineering cost used to be, and a filter runs on evidence.
Product management job did not shrink to writing tests but It moved to the question underneath them: does this make sense? Call it make-sense product management, and two terms carry it.
Make-sense check = the evidence that a feature is worth putting in front of users, solves a real problem and also works well enough on real inputs, and most importantly it moves a number the business cares about once failures are counted.
Pass bar = the line, written down in advance, that the evidence has to clear.
“Does this make sense?” is not a gut feeling question, but it is based on three
User: is the problem real and frequent? Evidence: behaviour, not a loud stakeholder.
Quality: does it work on real inputs? Evidence: a pass rate, not a demo.
Business: does it move a tracked metric after you count failures? Evidence: the metric, not “they’ll love it.”
Takeaways
Define the quality bar before you see any output. After the demo, “good enough” bends to whatever the prototype already does. A bar written after is just a description of the demo —> remember when everyone was talking about test-driven development?
FEATURE: AI reply drafts, support inbox
USABLE: Agent sends it with under 1 min of edits
MUST NEVER: - Promise a refund the policy forbids
- Draft anything on a legal threat
PASS RATE: 70% of real tickets
METRIC: Agent handle time
WRITTEN: Before the first demoRun a fixed set of real cases and slice them. 20/30 isn’t “almost there.” It’s one slice ready, one close, two dangerous, averaged into a number that describes none of them.
Then surface your business sense: is it worth it? You can foresee the impact by counting value per successful task:
Value = pass rate × gain − (1 − pass rate) × cost of a failure
So is the eval the new PRD? Only if it carries the decision and not just a set of checks: the user problem, the must-nevers, the bar and why it’s 70 and not 90 and what evals add on top is that it survives every rebuild, and reruns against every new model.



