Loading tutorials…
Loading tutorials…
Unbounce makes A/B testing easy — too easy. Most marketers ship 'A/B tests' that lack statistical significance, test trivial changes, and conclude prematurely. Here's how to test like a CRO professional.
Who this is forMarketers running paid ads on Unbounce landing pages with 100+ conversions/month. If your page has under 100 conversions/mo, A/B testing isn't worth the time yet — focus on traffic + offer first.
What you'll need
Step 1
Calculate sample size needed. Don't start testing without 100+ conversions/month on the control page.
Sample size calculator: use Optimizely's free tool or Unbounce's built-in stat sig calculator.
Inputs needed: baseline CR (current page CR), minimum detectable effect (typical: 10-20% lift), confidence level (95% standard).
Output: required conversions per variant. Typical: 100-1,000 conversions per variant for meaningful results.
If your page generates < 100 conversions/month: A/B testing isn't useful. Focus on driving traffic + improving offer first.
If you have 100-300 conversions/month: test ONE thing at a time, give each test 2-4 weeks.
If you have 500+ conversions/month: more aggressive testing OK, multi-variant testing possible.
Step 2
Not 'change button to red.' Real hypothesis: 'Replacing generic headline with audience-specific headline will lift CR by 10%.'
Hypothesis format: 'Changing [X] to [Y] will [result] by [magnitude] because [reason].'
Good hypothesis: 'Adding 3 testimonials below the hero will lift CR by 15% because new visitors lack trust signals.'
Bad hypothesis: 'Change button color to red.' (Why? Trivial change, unclear mechanism, likely no measurable effect.)
Test things that matter: headlines, subheads, social proof position, form field count, hero image, CTA copy, pricing display, value props.
Don't test things that don't matter: button color, font choice, color palette, image position by 10 pixels.
If you can't articulate a reason the change should affect CR, don't test it.
Step 3
Unbounce → Pages → click your page → A/B Test → New Variant. Duplicates the page; edit the one thing you're testing.
Pages → click the page → top right → 'A/B Test.'
Click 'Add New Variant.' Unbounce duplicates the page. Name the variant clearly: 'V2 — Headline test 2026-06-01.'
Edit ONLY the thing you're testing. Don't change the headline AND the CTA AND the hero image — you won't be able to tell what drove the result.
Common test types: Headline rewrite (high impact), subhead rewrite (medium), hero image change (medium), form field count change (high), CTA copy change (medium), social proof position (medium).
Save the variant.
Step 4
A/B Test → Traffic Distribution. Default 50/50 is fine for binary tests. 33/33/33 for three variants.
A/B Test panel → Traffic Distribution → set per-variant percentages.
50/50: standard for control + 1 challenger.
33/33/33: for 3 variants (1 control + 2 challengers). Less statistical power per variant.
Smart Traffic OFF for testing: if you have Smart Traffic enabled, turn it off for the duration of the test. Smart Traffic dynamically routes traffic, which breaks statistical assumptions of an even A/B test.
Start the test: click 'Start Test.' Unbounce begins splitting traffic.
Note the start date. You'll want to run for at least 2 full weeks (to capture day-of-week variance).
Step 5
Run until you hit required sample size AND at least 2 full weeks. Don't stop early just because one variant is winning.
Day-of-week variance: traffic and CR vary by weekday. Mondays are different from Saturdays. 1 full week minimum; 2 weeks better.
Sample size: per your earlier calculation. Common: 200-500 conversions per variant.
Resist the urge to peek and stop early. 'Variant B winning' at 80 conversions could flip at 200.
Resist the urge to launch a second test mid-test. Keeps results clean.
Communicate with team: tell paid media to keep ad spend stable during the test. A sudden 2x in traffic mid-test biases results.
If you have a strong intuition the variant is losing (CR -30%+ vs control): you can stop early. Cutting losses fast is fine. Stopping a winner early is the problem.
Step 6
A/B Test panel → Statistical Significance. Unbounce shows % confidence. Wait for 95% before declaring a winner.
A/B Test panel → variant stats: visits, conversions, conversion rate, lift vs control, statistical significance.
Statistical significance = probability the result isn't random chance. 95% is the standard threshold.
Under 95%: keep running. Over 95%: you have a valid result.
If lift is positive AND significant: deploy the winner.
If lift is negative AND significant: keep the original (variant lost).
If lift is positive but NOT significant: more data needed. Run longer.
If lift is negative but NOT significant: more data needed. Don't declare loser yet.
Don't trust your gut over the math. Stats win in the long run.
Step 7
Win or lose, document. Failed tests teach you what doesn't work. Compound learning over time.
Create a testing log: spreadsheet with one row per test.
Columns: test name, hypothesis, control CR, variant CR, lift %, statistical significance, start date, end date, sample size, conclusion, learning.
Even 'no winner' tests teach: 'Trying audience-specific headline didn't lift CR — audience may already be qualified before landing.'
Plan next test: pick the next-highest-impact thing on your list. Don't run multiple tests on the same page simultaneously.
Compound over time: 5-10 tests per quarter = 50-100 tests over 2 years = real CR optimization. Without a log, learnings vanish.
Common mistakes
Testing without enough traffic
What goes wrong: Test on a page with 30 conversions/month → conclude after 50 total conversions → result is random noise → "win" doesn't replicate.
How to avoid: Calculate required sample size first. Under 100 conversions/month = skip A/B testing entirely. Focus on traffic + offer.
Stopping tests early when one variant is winning
What goes wrong: Variant B leads after 50 conversions → marketer declares victory → deploys → CR doesn't lift in reality. False positives are very common at small sample sizes.
How to avoid: Wait for statistical significance (95%) AND minimum sample size AND 2+ full weeks. Resist the peek-and-stop urge.
Testing multiple changes simultaneously
What goes wrong: Variant changes headline AND hero AND CTA. If it wins, you don't know which change drove the win. Can't replicate.
How to avoid: One change per variant. Isolate the variable. If you want to test multiple, use multivariate testing (Optimize plan) — but it needs 10x more traffic.
Testing trivial changes (button color)
What goes wrong: Button color rarely moves CR more than 1-2%. Required sample size to detect 1% lift: 5,000+ conversions per variant. Months of testing for nothing.
How to avoid: Test high-impact elements: headline, offer, form fields, social proof, hero image. Save button color tests for later if at all.
Running Smart Traffic during A/B tests
What goes wrong: Smart Traffic dynamically routes high-converting users to one variant. This biases the test — the 'winner' isn't actually better, it just got the better visitors.
How to avoid: Disable Smart Traffic during A/B tests. Re-enable after declaring a winner.
Not documenting test results
What goes wrong: Run 50 tests over 2 years. Can't remember which tests won, lost, or why. Same hypothesis gets re-tested. Learnings don't compound.
How to avoid: Maintain a testing log spreadsheet. One row per test. Hypothesis, results, conclusion. Reference before launching new tests.
Recap
Done — what's next
How to build a high-converting Unbounce landing page
Read the next tutorial
Hand it off
A/B testing on Unbounce looks easy, drives real CR lifts only when done correctly. Most DIY testing programs accidentally hurt CR by acting on noise. A CRO specialist at $14-16/hr can build a real testing roadmap + run 4-6 tests in a quarter, typically $1,000-2,000 total — pays back via the first 10-20% CR lift.
See specialist rates
Minimum 2 full weeks (for day-of-week variance). Also: until you hit 95% statistical significance + required sample size. Most tests run 2-6 weeks.
Median: 5-15% lift on winning tests. Top tests: 30-50% lift on big-hypothesis changes (offer, headline, form). Over a year: 30-100% cumulative lift on a well-tested page.
Yes — Build plan supports A/B testing. Smart Traffic is Experiment plan and above. For pure A/B (manual analysis), Build is enough.
Not on the same page at the same time. They conflict. Run A/B tests to find a winner. Then enable Smart Traffic to dynamically optimize the winner against future variations.
2 (control + 1 challenger) for most cases. 3 max — you split traffic, need more samples per variant. Multivariate testing (4+ combinations) requires 1,000+ conversions per variant — most marketers don't have enough traffic.
Unbounce
Unbounce's drag-and-drop builder makes shipping a landing page in 3 hours feel doable. The harder question: does the page actually convert? Most DIY pages launch at 1-3% CR. Specialists routinely ship pages at 8-15%.
Unbounce
Smart Traffic is Unbounce's machine-learning router — it analyzes visitor attributes and sends each one to the variant most likely to convert THEM. Typically lifts CR 10-30% vs even A/B testing. The catch: it needs enough traffic to learn.
Unbounce
Page launched, traffic flowing, conversions flat. The diagnostic sequence below is what a CRO specialist runs in their first 90 minutes on the account. Usually 1-2 of these unlocks the CR lift you've been chasing.
Unbounce
Unbounce can pay back its $99-625/mo fee 5-20x with the right setup. Or break even with mediocre setup. The gap is where specialists earn their fee.