How do I use AI on customer work without it sending something wrong?

By Ranjit Bhinge, founder of Blur Studio · Updated · 6 min read

Short answer: Put a person between the AI and the customer, and size that check to the stakes and the volume. "Human in the loop" means a person reviews, approves or can stop what an AI system does. For anything that can cost money, upset a customer or create a legal problem, a person approves every output. For routine, low-risk work, check a sample, for example 1 in 10. For everything the AI is unsure about, send it to a queue that a named person clears each day.

Ranjit Bhinge

You want AI to do customer-facing work, and you want a rule for where the person checks so nothing wrong goes out.

I'm Ranjit, and I run Blur Studio: 15+ years building software, 120 apps taken live, and clients who have gone on to raise $10M+. I build AI systems and processes that help businesses do a lot more with a lot less time. I also work differently from most of what else is out there, more on that further down this page. Say hello.

What "human in the loop" means

It means a person is part of the process, not just at the start or the end. The AI does the first pass, such as drafting a reply, sorting a request or filling a form. A person reviews, changes, approves or rejects the result before it takes effect. The aim is simple: AI makes the work faster, and the person keeps responsibility for what leaves the building.

The need is real. OWASP, a nonprofit that publishes security guidance, lists "excessive agency" among the main risks of systems built on large language models (the technology behind ChatGPT and Claude). It means the system has more power than it needs, so a wrong or manipulated output can cause real damage. OWASP's advice for high-impact actions is to keep a person in the loop, for example by requiring user approval before an app publishes a social media post. The European Union's AI Act, in Article 14 on human oversight, adds a warning for the person doing the checking: automation bias, the habit of trusting the system's output too easily. A check that only clicks "approve" is not a check.

Decide by stakes and volume

Two questions set where the check goes.

  1. Stakes: if this output is wrong and reaches the customer, what does it cost? A wrong opening hour is small. A wrong price, a promise you cannot keep or advice about health, money or law is large.
  2. Volume: how many outputs a day? Ten a day can all be read. Four hundred cannot.
Low volume (under about 30 a day)High volume (over about 30 a day)
High stakesApprove every output. Draft only, a person sends.Approve every output with a second person on the hard ones, or limit what the AI may do until you can staff it. Do not sample.
Medium stakesApprove every output for the first month, then sample.Sample 1 in 5 to 1 in 10, plus an exception queue.
Low stakesSample, or review after sending.Sample 1 in 10 to 1 in 20, plus an exception queue.

The 30-a-day line and the sampling rates are my own rules of thumb for a small team. Change them to fit your staff, but write the rule down.

Three patterns that work

1. Draft and approve

The AI writes a draft and a person reads it and presses send. Use it for customer emails, quotes, proposals, social posts and anything involving money or promises. For email, tools such as the ones in my guide to an AI email assistant work this way: they draft, you send.

Review time is the cost to plan for. As a made-up example: writing a customer reply from scratch takes 5 minutes, and reading and approving an AI draft takes 1 minute. For 20 replies a day that is 100 minutes against 20, a saving of 80 minutes a day, but you still spend those 20. Time your own work for a week to get your real figures, and cost the minutes properly: the US Bureau of Labor Statistics reports that wages were 70.0 percent of private-sector employer compensation costs in June 2026, so an hour of staff time costs more than the wage.

2. Sample check

The AI acts on its own and a person reads a random sample afterward. If you sample 1 in 10 and the AI handles 100 items a day, a person reads 10. If the sample shows more than a small number of problems, you move back to approving every item. Use it for low-stakes work, such as tagging, sorting, summaries for internal use and answers to simple questions from your own published information.

One warning: sampling finds patterns, not every error. If one wrong item in a hundred would be a disaster, do not use sampling for that kind of item.

3. Exception queue

The AI handles what it is confident about and passes the rest to a person. The rule has to be written down: it passes anything that mentions a refund, a complaint, a legal word, an unusual amount, a customer marked as important, or something it could not match to your information. This is the pattern for AI that talks to customers, such as an AI customer service agent or a phone agent. Before customers hear it, run through the checks in my guide to testing an AI phone agent.

The queue only works if someone clears it. Give it an owner and a time limit, such as "every item answered within 2 working hours", and a fallback when the owner is away.

The named-approver rule

For every AI workflow, write down one named person who is responsible for what goes out, and one person who covers when they are off. "The team checks it" means nobody checks it. The approver needs three things:

  • the authority to reject an output and stop the workflow;
  • enough time, which means the review time is counted in their day and not added on top;
  • the context to judge it: your prices, policies and customer history.

This is the practical form of what Article 14 asks of overseers: understand the system's limits, watch for odd behaviour, be able to override the output and be able to stop the system.

Making the check a real check

  • Show the source. A draft that says where each fact came from is faster to verify than one that does not.
  • Plant test items. Now and then, put a known-wrong draft in the queue. If the approver passes it, you have found automation bias.
  • Log edits and rejections with a one-line reason. After a month the log tells you what to fix in the instructions.
  • Set a pace limit. A person who approves 200 items an hour is not reading them.
  • Limit the AI's power. It should draft, not send; suggest, not delete. Give it only the access the task needs, which is the OWASP advice on excessive agency.

When to loosen review

Loosen review in steps, never all at once. My rule: after you have approved at least 100 items of one kind with almost no edits (and no serious errors), move that kind to sampling at 1 in 5. After another month of clean samples, move to 1 in 10. Keep exceptions and high-stakes items on full approval for good. Tighten again right away if the sample shows a problem, if you change the instructions, or if the AI vendor changes the underlying model.

To decide if a change is worth it, put the saved minutes against the risk. My guide on calculating the ROI of AI automation shows how to count review time in the cost.

Examples by type of work

WorkPatternWhy
Quotes and proposalsDraft and approvePrices and promises bind you
Customer emailsDraft and approve, then sample for routine onesTone and accuracy matter
Inbox sorting and taggingSample checkA wrong tag is cheap to fix
Booking and opening-hours questionsSample check, exception queueAnswers come from your own information
Complaints, refunds, legal or health topicsAlways a personHigh stakes, emotional, sometimes regulated
Phone answeringException queue with a call-backCallers cannot see a draft, so test first

The wider picture is in AI implementation and AI workflow automation for small business. If you want AI that does more on its own, read what an AI employee is first, and what an AI agent is, because more independence means a stronger case for review. NIST's AI Risk Management Framework, a voluntary US standards-body guide, organises AI risk work into four functions: govern, map, measure and manage.

How I'd approach this

I start every customer-facing AI workflow at full approval and earn my way down. I would write the rule on one page: what the AI may do, what a person must approve, what goes to the queue, who the approver is and who covers. I would count review time as a real cost from day one, so nobody is tempted to skip it when things get busy. I would not let AI send anything involving money, promises or sensitive subjects without a person, however good the demo looks.

When I build these for clients, the approval step, the queue and the log are part of the design from the start, not added after something goes wrong. Fixes and finishes start around $1,500 if you already have a workflow that needs its checks added.

Questions people ask

What is human in the loop in AI?

It means a person reviews, approves or can stop what an AI system does before it takes effect. The AI does the first pass and the person keeps responsibility for the result.

Does human in the loop slow everything down?

It slows the last step, but reading a draft is usually much faster than writing one. Measure your own review time for a week. The saving is smaller than the vendor's demo and still real for most repeat work. See how to calculate the ROI.

How often should a person check AI output?

For high-stakes work, every item. For medium or low stakes, start with every item for a month, then sample 1 in 5 and later 1 in 10 if the samples stay clean. These are my rules of thumb, not a standard.

What is an exception queue?

A list of items the AI hands to a person: anything it is unsure about, plus anything on your written trigger list, such as refunds, complaints or unusual amounts. It only works if someone owns it and clears it daily.

Can an AI agent work with a human in the loop?

Yes, and it should for anything with real consequences. An AI agent is software that takes steps toward a goal on its own, so decide which steps need approval. A common rule is that it may draft and prepare, and a person approves anything that sends, pays or deletes. See what an AI agent is.

Who should be the approver?

One named person who knows your prices, policies and customers, has the authority to reject outputs, and has review time built into their day. Name a backup too.

Is human oversight a legal requirement?

For high-risk systems the EU AI Act requires human oversight (Article 14), with dates for compliance that depend on the type of system. For most small business uses it is good practice, not law, but rules vary by country and industry, so ask a lawyer if you work in a regulated field.

More in this topic: Problem solving: AI strategy and adoption (24 guides) →

Sources

Ready for the next step?

A 30 minute call. If you do not need me, I will tell you.

Ranjit Bhinge You talk to me, Ranjit. I reply within a few hours.

Not sure what you need? Tell me what is going on. I will point you the right way, even if it is not me.

Blur Studio · 30 minute call

Book a 30 minute call

You talk to me, Ranjit. If you do not need me, I will tell you.

  • We talk about your business and what is going on.
  • You leave with a clear next step, even if it is not me.
  • I reply to every message within a few hours.
Loading the calendar…