How to run a website test without losing bookings

By Ranjit Bhinge, founder of Blur Studio · Updated · 12 min read

Short answer: To test a website change without losing bookings, change only the new version, keep the phone number and booking link working and identical on both, check both versions on a phone before any visitor sees them, start with a small share of traffic, and write a stop rule before you launch. A guardrail is a second number you watch, such as bookings or calls, that tells you when to switch the test off. Tools such as Unbounce, a landing page builder, and Ploy, an AI platform that builds and runs business websites, let you set one, but you can do it by hand too.

Ranjit Bhinge

A test should cost you a little traffic, not a week of bookings. Here are the checks, stop rules and rollback steps that keep the risk small.

I'm Ranjit, and I run Blur Studio: 15+ years building software, 120 apps taken live, and clients who have gone on to raise $10M+. I build AI systems and processes that help businesses do a lot more with a lot less time. I also work differently from most of what else is out there, more on that further down this page. Say hello.

The main risk in a website test is not that the new version loses. It is that the new version is broken and nobody notices. A button that does nothing on a phone, a form that sends to the wrong inbox, or a phone number that is not tappable can cost a service business a week of calls before anyone looks. So the plan is: check before launch, start small, watch the one number that matters, and know exactly how to switch the test off.

This page is the safety guide. For choosing what to change and how to roll out version B, read how to try a new version of a weak ad landing page safely. For showing a client the result, read how to prove to a client that a page change helped. Prices and features here are as of November 2026, so check each vendor page before you pay.

This page may contain affiliate links from which Blur Studio earns commissions.

What actually goes wrong in tests

  • A broken path. The form, the phone link or the booking button fails on the new version, often only on phones.
  • Silent tracking loss. The new version does not record the goal, so it looks like it converts worse than it does, or you cannot tell at all.
  • Too much changed at once. A new headline, price, layout and form together make it impossible to know what helped or hurt.
  • Stopping too early or too late. People either kill a test after a bad hour or let a clearly failing one run for weeks.
  • Search or ad rule trouble. A badly set up redirect or hidden variant can confuse search engines or break ad-platform rules.

Each one has a cheap check, and the sections below go through them.

The pre-launch check (do this before any visitor sees version B)

Run the checks on version B and on version A, on a phone and on a laptop.

  1. Submit the form on both versions. Use a real phone. Confirm the message arrives in the right inbox or system, and the thank-you page or message shows.
  2. Tap the phone number on both. It should open the dialler with the right number. If you use call tracking numbers, check each version shows the number you expect.
  3. Click the booking button on both. Make a test booking and cancel it. Check it appears in your calendar or booking system.
  4. Check the goal is recorded. Look in your analytics (for example Google Analytics 4, Google's free web analytics tool, where an important action is called a key event) or your testing tool for the test goal. If it does not show, fix that first.
  5. Test speed on a mobile connection. Not office wifi. If B is clearly slower, fix the image or script before launch.
  6. Check both on a small phone and a large one. Make sure no button is hidden behind a pop-up, chat widget or cookie banner.
  7. Save a copy of version A. Know exactly how to get back to it.

The most important rule is to keep the booking path the same on both versions unless the booking path is the thing you are testing. Same phone number, same booking link, same form destination. That is how you keep a test from becoming a gamble.

Set a guardrail and a stop rule

Your test has a goal, the number you hope will rise. A guardrail is a second number that must not fall. For example, the goal could be "form sends" and the guardrail "booked calls" or "bookings per 100 visits". Without a guardrail, a version could win on form sends by promising something you cannot deliver, and lose you real bookings.

BusinessGoalGuardrailExample stop rule
Emergency plumberCalls and formsCalls answered and booked jobsStop B if jobs per 100 visits fall below half of A's after 100 visits each
Dental or clinicBooking form sendsAppointments keptStop B if confirmed appointments from B are under half of A's per visit, after a set number of visits
Salon or restaurantOnline bookingsBooking errors and refundsStop B at once if any booking fails to arrive
Online shopAdd to basketCompleted ordersStop B if completed orders per visit drop below A's by a set margin after enough visits

Write two kinds of rule:

  • Break rules, which apply at any time. Stop at once if the form, phone link, booking flow or tracking fails. No statistics needed.
  • Performance rules, which apply after a minimum number of visits. A bad hour on 15 visits proves nothing. Set a minimum number (for example 100 visits per version) before the performance rule can trigger, then stop B if it is far behind on the guardrail.

Use a generous margin for the performance rule, such as "less than half", because small samples swing wildly. A stop rule is for protecting bookings, not for picking the winner.

Start small, then widen

  1. Days 1 to 3: 10 to 20 percent to version B. This is the smoke test. You are looking for breakage.
  2. Check every day. Look at form sends, calls and bookings for both versions. Compare against the number you would expect for that share of traffic.
  3. If clean, go to an even split. A 50/50 split reaches a result fastest. If this page is most of the client's business, stay at 20 to 30 percent and accept a longer test.
  4. Do not stop and start. Changing the split mid-test can skew results unless you track it.
  5. Leave it until the sample is reached. Set the sample first. Evan Miller, a writer on testing, found that stopping the moment a result looks significant gave a false winner 26.1 percent of the time in his example, against the 5 percent expected. Peeking for breakage is fine. Calling a winner early is the mistake.

How many visitors is enough? These figures are derived by me, not from a vendor. I used the standard two-sided formula for comparing two rates, with equal groups, 95 percent confidence and 80 percent power: moving from a 5 percent rate to 7.5 percent needs about 1,500 visitors per version, and from 5 to 6 percent needs about 8,200. Use a calculator such as Evan Miller's for your numbers. For duration, Optimizely, a large testing company, says at least one full business cycle, typically seven days. Convert, another, says one to two full cycles. They differ, and so do their significance levels (Optimizely usually 90 percent, Convert 95 percent as its reference), so pick a rule and state it. For a booking business with weekday calls and weekend forms, two full weeks is a sensible floor.

A worked example with made-up numbers

These numbers are invented to show the arithmetic. They are not results from a real business.

A physiotherapy clinic sends 900 ad clicks a month to a booking page. About 45 become booked first appointments, a rate of 5 percent. The clinic wants to test a shorter booking form (six fields to three). Goal: form sends. Guardrail: first appointments that the clinic confirms by phone.

  • Before launch. The owner submits both forms from a phone, checks the emails arrive, and confirms a test booking appears in the calendar. The phone number is the same on both pages.
  • Days 1 to 3, 20 percent to B. About 18 visits reach B (900 clicks a month is about 30 a day). One or two forms arrive, and all reach the inbox. Nothing is broken.
  • Week 1 onward, 50/50. Stop rule: if after 100 visits B has fewer than half the confirmed appointments A has, switch B off.
  • At 100 visits each. A: 6 confirmed appointments. B: 9 form sends but 4 confirmed appointments. B is not below half of A, so it continues, but the owner notes that more forms came with fewer confirmations.
  • End of week 4. About 450 visits have reached each version. That is far below the thousands the formula asks for, so the owner cannot call a winner on rates. But B has more form sends and about the same number of confirmed appointments. The shorter form got more forms but not more patients, so the owner keeps the longer form for now. The guardrail stopped the owner from treating form sends as a win.

The lesson is the guardrail. Measured only by form sends, B would have looked like a winner.

Know what your numbers leave out

Every testing tool counts only the visitors it can see. Some visitors refuse cookies, some block analytics scripts, and some bookings happen by phone and never touch a script. So compare the tool's count with the real records, such as your booking system, form inbox and call log, at least once during the test. If the tool shows 40 bookings and the booking system shows 55, you know the tool is missing some, and the same gap may apply to both versions. Ploy's analytics, for instance, follow each visitor's cookie choice, and its experiments do not count visitors who refuse cookies, who always see the current version. Check local privacy law and read is website visitor tracking allowed under privacy law. Do not assume any tool is immune to ad blockers.

For phone leads in particular, make sure each version can be told apart, for example with a separate tracking number. How to track which ad made the phone ring shows how.

Keep it safe for search engines and ads

If your test pages can be found by search engines, Google's guidance on testing is short:

  • Never show Googlebot different pages from the ones people see. Google calls that cloaking, and it breaks its spam policies even during a test.
  • Use rel="canonical" on alternate URLs. Point the test variants at the original.
  • Use a 302 redirect, not a 301, if you redirect to a variant. A 302 is temporary.
  • Run the test only as long as you need. Remove the test elements promptly afterwards. Google says a test left running too long may look like an attempt to deceive search engines.

Check Google's current guidance before you launch. Paid ad pages also have platform rules, so check the destination rules for your ad platform. For deeper setups that serve variants under your main domain, ask a developer, because mistakes here are hard to undo.

Your options, from least to most effort

OptionWhat it isCost (as of November 2026)Guardrails
No tool: one after the otherPage A for two weeks, then page B$0 beyond your timeYou watch your own bookings. Weeks differ, so results are rough.
UnbounceLanding page builder with built-in A/B testing, free hosting includedStarter $29, Build $99, Experiment $149, Optimize $249 a month billed monthly; Agency from $349 a month billed annuallyUnlimited A/B testing, manual traffic allocation and conversion reporting are listed from Experiment. Starter has 5 pages and does not list testing.
Instapage and HubSpotA page builder for ad teams, and a marketing platform with a page builderInstapage: Create $79 a month billed annually or $99 monthly; Optimize $159 annually or $199 monthly; Convert custom, by demo. HubSpot: per seat, Professional $90 a month billed annually, $100 monthlyInstapage lists traffic splitting on Optimize. For HubSpot, check the page for which plan includes testing
GrowthBookOpen-source tool for experiments and feature flagsStarter free for up to 3 users; Pro $40 per seat a monthNeeds a developer to set up
PloyOne section of one page against a challengerFree $0, Starter $50, Pro $300 a month, Enterprise by quoteOne guardrail in the form, up to three through chat

Google Optimize, once the free option, stopped on 30 September 2023. Optimizely and VWO do not publish prices for web testing. See best landing page builders for ads for a wider comparison.

Where Ploy fits (one option of several)

With Ploy, you can run an A/B test on one section of a live page, with a goal and a guardrail. In Ploy's docs the experiment form takes one guardrail, and you can add up to three by chat with Ploy's Agent. You set a traffic split between 10 and 90 percent for the challenger, and the test can be paused and resumed. The challenger is drafted in a chat, shown beside the live version in a compare view, and "Start experiment" publishes it and checks the live page. These steps match the safe rollout: start at 10 to 20 percent, pause if something looks wrong, then raise the share.

Requirements and limits to check first:

  • It is on request. Email sales@ploy.ai to have experiments switched on for your workspace.
  • The site must be published and Ploy analytics on. Your goal events must already be tracked, so run the pre-launch check above first.
  • One section of one page per test. The docs do not describe whole-page splits.
  • A plan limit applies to running experiments, counted across the organisation. The number is not published, so ask.
  • Visitors who refuse cookies always see the current version and are not counted.
  • The statistics method and significance level are not published. Use your own sample and stop rules.

Ploy's pricing page shows Free $0, Starter $50 a month, Pro $300 a month and Enterprise by quote, as of November 2026, with credits that, per Ploy's docs, do not carry over. Read Ploy pricing explained before you pick a plan, and have a person read any AI-drafted copy before it goes live.

Who should skip Ploy for this

  • Your analytics and goals are not set up. Ploy needs analytics on and goal events already tracked. Fix tracking first, with any tool.
  • You need a whole-page or whole-flow test. A dedicated page tool such as Unbounce fits better. See Ploy vs Unbounce.
  • You need the booking flow itself to be tested. A tool that changes one page section will not cover a third-party booking system. That needs a custom build.
  • Traffic is very low. No tool will give a clean answer. Use a careful before and after.

Rollback plan and test-day checklist

Write your rollback before you launch, so no one has to think when something breaks.

  1. Who can switch version B off, and how (pause the experiment, send 100 percent to A, or unpublish B).
  2. Where version A is saved.
  3. Who checks the form inbox, calls and bookings, and how often in the first week.
  4. The break rules (any failure of form, phone, booking or tracking means stop).
  5. The performance rule (minimum visits, margin, guardrail).
  6. The end date and who removes the test setup afterwards.

Copy-ready test-day checklist: form sent and received on both versions; phone link tapped on both; test booking made and cancelled; goal and guardrail recorded; page speed checked on mobile data; version A saved; start date outside holidays and sales; split set to 10 to 20 percent; rollback owner named; first check booked for tomorrow morning.

How I'd approach this

I would treat a test like a small change to a live shop, not like a science project. The aim is to lose as little as possible while you learn. That means the booking path stays the same, version B starts with a small share, one person is named to watch the numbers, and the rule for switching it off is written down.

First-week plan: day 1, write the goal, the guardrail and both stop rules. Day 2, build version B and run the pre-launch check on a phone. Day 3, start at 10 to 20 percent. Days 4 and 5, check form sends, calls and bookings each morning. Day 6, move to an even split if all is clean. Day 7, check again and leave it until the sample is reached. Decision rule: if you need a quick way to run a section test with a guardrail, ask for experiments on Ploy and keep your own stop rules. If the page is your main source of bookings and traffic is low, change one thing at a time and compare before and after.

If you need a test that is wired into your own booking or order system, so each booking records which version produced it, Blur Studio builds that custom version. See what that includes.

You may also want testing a new product line page before a full rebuild.

Questions people ask

How do I test a website change without losing bookings?

Keep the phone number, booking link and form destination identical on both versions, check both on a phone before launch, send only 10 to 20 percent of traffic to the new version at first, and write a stop rule in advance. Watch bookings, not just form sends.

What is a guardrail in an A/B test?

A guardrail is a second number that must not get worse while you chase the main goal. For example, the goal is form sends and the guardrail is confirmed appointments. If the guardrail drops sharply, you stop the test. Ploy's experiments accept one guardrail in the form and up to three through chat.

When should I stop a test early?

Stop at once if the form, phone link, booking flow or tracking breaks. For performance, set a minimum number of visits first, such as 100 per version, then stop only if the new version is far behind on your guardrail. Do not stop early because it looks like it is winning.

How long should a website test run?

Until you reach the sample you set in advance and for at least one to two full business cycles. Optimizely says at least one cycle, typically seven days, and Convert says one to two. For a booking business, two weeks is a sensible floor. See how to try a new version safely.

Will an A/B test hurt my search rankings?

It should not if you follow Google's guidance: no cloaking, rel="canonical" on alternate URLs, a 302 (temporary) redirect, and remove test elements when the test ends. Check Google's current guidance before launching.

Does Ploy let me stop a test if bookings fall?

Ploy's docs say an experiment can be paused and resumed and takes a goal plus a guardrail. They also say Ploy warns when the challenger is very likely doing worse, but they do not say it stops a test for you automatically, so assign someone to watch it. Experiments are on request through sales@ploy.ai, and the statistics method is not published. See Ploy for current details.

Can I test without a tool?

Yes. Run version A for two weeks and then version B for two weeks, watching your own booking records, and compare. It is rough because weeks differ, but it carries little risk. See how to prove a page change helped for how to present the result.

More in this topic: Websites: testing and proving page changes (4 guides) →

Sources

Ready for the next step?

A 30 minute call. If you do not need me, I will tell you.

Ranjit Bhinge You talk to me, Ranjit. I reply within a few hours.

Not sure what you need? Tell me what is going on. I will point you the right way, even if it is not me.

Blur Studio · 30 minute call

Book a 30 minute call

You talk to me, Ranjit. If you do not need me, I will tell you.

  • We talk about your business and what is going on.
  • You leave with a clear next step, even if it is not me.
  • I reply to every message within a few hours.
Loading the calendar…