When AI Should Suggest, Not Decide

A useful question for any AI feature is not "Can the model do this?"

It is "Who should carry the consequence when it is wrong?"

That question changes the design. An AI system can draft a reply, rank applications, flag an unusual payment, summarize a patient note, or recommend which customer needs attention. Technically, it may also be able to send, reject, freeze, publish, or prescribe. The extra verb is where assistance becomes authority.

Sometimes that authority is appropriate. Many times the better product stops one step earlier. It gathers the evidence, proposes an action, explains the uncertainty, and leaves a person to decide.

This is not timidity about AI. It is a practical way to use a probabilistic system inside work where context, exceptions, and accountability still matter.

Suggestion and decision are different products

Imagine an inbox assistant reading a customer message.

As a suggestion tool, it can identify the order, summarize the complaint, retrieve the refund policy, draft a response, and point out that the customer has contacted support twice. An employee reviews the facts, adjusts the tone, and sends the message.

As a decision tool, it can approve the refund and send the reply without review.

The second version is not merely the first with one less click. It needs a clearer policy, reliable identity matching, permission limits, transaction safeguards, an audit trail, reversal tools, and a plan for the case where the customer's situation is not covered by the examples the system learned from.

Products get into trouble when they treat that final action as a convenience setting. "Auto-send" sounds small in a menu. It changes who has authority.

Keep a person in the loop when the cost is uneven

Not every error has the same shape.

If an AI reorders a private reading list badly, the cost is mild and reversible. If it rejects a job candidate, blocks a customer account, or gives a confident health instruction, the person affected carries a much larger cost than the organization saving a few minutes.

Human review is especially valuable when:

• The outcome affects money, access, safety, employment, or reputation.
• The action is difficult to reverse or the person may never know it happened.
• Important context lives outside the data available to the system.
• The correct choice depends on policy, empathy, or an exception.
• The model cannot show enough evidence for a reviewer to judge the suggestion.
• Errors may systematically burden one group more than another.

The phrase "human in the loop" can still hide a weak design. A reviewer who must approve 700 suggestions before lunch is not exercising judgment. They are providing a signature. Oversight only works when the person has time, context, authority to disagree, and a clear way to correct the system's proposal.

Show the evidence, not just the answer

A confidence score is not an explanation. "92%" looks precise while saying very little about which fact the system used or what it may have missed.

Useful review screens put the source beside the suggestion. Highlight the lines in the policy that support the draft. Show which customer record was matched and why. Display the original message without forcing the reviewer to open another tool. If sources disagree, show the disagreement.

The reviewer should be able to answer three questions quickly:

1. What is the system proposing?
2. What evidence led it there?
3. What happens if I accept it?

This design improves more than safety. It makes the feature faster to use because the person does not need to reconstruct the context from scratch.

It also reveals when AI is adding little value. If the reviewer must reopen every source, redo the reasoning, and rewrite the result, the feature may be producing ceremony rather than assistance.

Give disagreement somewhere to go

Many AI interfaces offer Accept and Regenerate. Neither captures a useful correction.

The system may have the wrong customer, an outdated policy, missing context, a tone problem, or a reasonable suggestion that the employee chooses not to follow. Those are different kinds of disagreement. Treating all of them as a thumbs-down wastes information and makes future improvement difficult.

Provide a small set of correction paths that match the work. "Wrong source," "missing context," "policy exception," and "edit before sending" can be more useful than a generic feedback box. Do not force the employee to write an essay for every correction. They have a job already.

Record enough to investigate patterns, but be careful about turning review into surveillance. If employees believe every disagreement will be judged as resistance to the new system, they will click Accept and fix the damage elsewhere. A feedback loop needs psychological safety as much as a database column.

Automation can earn more authority over time

Suggest-first does not have to mean suggest-forever.

Start by observing the work. Compare suggestions with actual decisions. Find the categories where reviewers consistently agree and the exceptions where they do not. Improve the data, policy, and interface. Then consider bounded automation for the low-risk cases.

A support tool might eventually send acknowledgements automatically while keeping refunds for review. A finance tool might categorize familiar recurring expenses but ask about unusual vendors. A publishing assistant might schedule approved content but never invent or publish a claim on its own.

The boundary can be based on value, confidence, category, customer state, or reversibility. Keep it understandable. A rule nobody can explain after an incident is not a reassuring rule.

Add an escape hatch too. People need to pause automation, undo an action, and see what happened. Monitoring should detect shifts in inputs, error patterns, and reviewer disagreement. A model that performed well last quarter is not a permanent fact about the world.

Preserve responsibility

The worst arrangement is one where the model makes the practical decision but a person remains nominally responsible. They did not have enough time or information to review, yet their name appears in the audit trail. That is accountability theatre.

If a person owns the decision, give them real control. If the system owns an automated action, the organization must own the system's behavior, including the recovery work. Do not hand the consequence to the nearest employee or the affected customer.

This principle also helps teams resist feature inflation. An AI assistant does not need to do everything to be valuable. Retrieving the right context, shortening a long document, spotting a missing field, or drafting a careful first version can remove substantial work while leaving judgment with the person closest to the situation.

We have written before about keeping a human in the loop and the real risks of AI. The practical version is simple: draw a line between preparation and authority for every feature. Decide who may cross it, under which conditions, and how the action can be corrected.

If you are planning an AI feature, bring us the decision it touches, not just the model you want to use. The best design may be the one that makes a person dramatically better informed before they press the button.