Most teams evaluate an AI feature with one number: how often is it right? It's the wrong question, and it hides the decision that actually matters.
Because two features can both be right 95% of the time and be worlds apart in whether you should ship them. What separates them isn't the 95. It's the 5 — specifically, what each wrong answer costs when it lands.
A wrong movie recommendation costs a shrug. A wrong medication dosage costs a life. Same accuracy on paper. Nowhere near the same product. If your design doesn't price the error, your accuracy number is lying to you.
Accuracy is a headline; error cost is the story
Aggregate accuracy averages away the thing you most need to see. It treats every mistake as one unit of wrong, when in reality your errors have wildly different price tags depending on which way they break and where they land.
The first split is direction. A false positive — the AI flags something that was fine — costs differently than a false negative, where it misses something that wasn't. A spam filter that quarantines a real invoice (false positive) and one that lets a phishing mail through (false negative) fail in opposite directions with opposite consequences. One annoys; one breaches. A model tuned to minimize total errors will happily trade the cheap one for the expensive one and call it an improvement.
The second split is placement. The same error costs more in some contexts than others. A wrong autocomplete in a casual note is nothing; the same wrong autocomplete in a legal clause is a liability. Position, not just correctness, decides the damage.
So before I look at how often a feature is wrong, I ask how expensively it's wrong — and in which direction. That reframing changes what you build.
Price the error before you build the feature
Here's the discipline I hold teams to. Before we design an AI feature, we write down the cost of each failure mode. Four questions, answered on purpose:
1. What are the failure modes? Not "it's wrong" — the specific ways. Wrong output confidently stated. Right output the user misreads. Refuses a valid request. Acts on a bad one. Name them.
2. Which direction is more expensive? For each mode, is the false positive or the false negative the one that hurts? You rarely get to minimize both. Decide which one you're willing to accept more of, because the model will make that trade whether or not you chose it consciously.
3. Is the error reversible? A draft the user reviews before sending is cheap to get wrong — they catch it. An email the AI sent autonomously is expensive — it's already gone. Reversibility is the single biggest lever on error cost, and it's a design decision, not a model property.
4. Who absorbs the cost? The user, a third party, or the business? An error that quietly harms someone who never consented to the AI is a different risk class than one the user brought on themselves.
Answer these and you're no longer designing for average correctness. You're designing for the worst plausible failure — which is the only number your users, and your lawyers, actually care about.
Design responses that match the price
Once you've priced the error, the product design almost writes itself. Low-cost, reversible errors get a light touch — let the AI act, make undo easy, move fast. High-cost or irreversible errors get friction on purpose.
The main levers, from cheapest error to most expensive:
- Autonomous action for errors that are cheap and reversible. The AI does it; the user corrects if needed.
- Review-then-commit when the error is expensive but a human can catch it. The AI drafts; a person approves. This is human-in-the-loop, applied where the cost justifies the friction.
- Confidence-gated behavior when cost varies by case. Act automatically when the model is sure and stakes are low; escalate to a human when it isn't or they aren't.
- Refuse or defer when a wrong answer is catastrophic and unrecoverable. The right move is sometimes for the AI to decline and hand off, not to guess.
Notice the pattern: the more a wrong answer costs, the more human judgment you insert between the model and the consequence. You are not slowing the product down arbitrarily. You are spending friction where it buys the most protection and spending none where it doesn't.
The failure I see most often is uniform design — the same interaction pattern bolted onto every AI action regardless of stakes. Autonomous everything, and the one catastrophic error slips through untouched. Or review-everything, and users drown in approvals for decisions that never needed them. Both come from skipping the pricing step.
Make it a required step, not an instinct
The reason this gets skipped is that it feels obvious in hindsight and invisible up front. Everyone agrees the medication AI needs more care than the movie AI. Almost nobody writes down how much more, or lets that number shape the design before the model is chosen.
So I make it a gate. No AI feature moves from concept to build until we've priced its errors and matched the design to the price. It takes an afternoon. It changes the model you pick, the interface you draw, and the failures you'll actually survive.
Accuracy tells you how good the model is. Error cost tells you how good the product is. Only one of those is your job.
FAQ
Isn't a more accurate model the real fix for error cost? It helps, but it never removes the question. Even a 99% model is wrong sometimes, and if that 1% lands on an irreversible, high-cost action, higher accuracy just makes the rare failure more surprising and less watched. You still have to design for the wrong answer — accuracy sets the frequency, design sets the consequence.
How do I price an error I can't put a number on? You don't need a precise figure. Rank them. Sort failure modes into cheap, expensive, and catastrophic, and note which are reversible. That ordinal ranking is enough to decide where autonomous action is fine and where you need a human in the loop. Precision comes later; the ranking is what changes the design.
What if I genuinely can't tell which error direction is worse? Then you've found the most important conversation to have before building, not after. Ambiguity about which failure is costlier usually means product, legal, and engineering are each assuming a different answer. Force the decision early — the model will make the trade-off for you if you don't.
Does adding friction for high-cost errors hurt adoption? Only if you apply it everywhere. Friction placed on the rare, expensive action is nearly invisible — users expect a pause before something irreversible. Friction placed on every action, including the trivial ones, is what kills adoption. Pricing errors is precisely how you put friction only where it's earned.
I'm Ravi Jadav, Chief Product Officer and Co-Founder at Sunbots Innovations and Co-Founder at Xwits Developers. Get in touch.