Web Experimentation
Why visual testing matters
Section titled “Why visual testing matters”Marketing teams generate hypotheses constantly — that a different headline would convert better, that moving the CTA above the fold would increase clicks, that showing social proof would build trust. Testing these hypotheses should not require a development sprint.
Web Experimentation lets marketers create and run tests directly on live pages using a visual editor. Changes are applied client-side through a JavaScript snippet — no code deploy, no developer dependency, no waiting for a release cycle.
How Web Experimentation works
Section titled “How Web Experimentation works”The snippet
Section titled “The snippet”Web Experimentation operates through a JavaScript snippet added to your site’s <head> tag. When a visitor loads a page:
- The snippet loads the experiment configuration
- It evaluates whether the visitor qualifies for any active experiments
- If yes, it modifies the page DOM to apply the variation
- It tracks visitor behavior (clicks, form submissions, page views)
- Results are sent to the Optimizely analytics backend
The snippet is designed to load early and apply changes before the page renders visually, preventing “flicker” — the brief flash where visitors see the original content before the variation appears.
The visual editor
Section titled “The visual editor”The visual editor lets you modify page elements without code:
| Capability | What you can do |
|---|---|
| Text changes | Edit headlines, body copy, button labels |
| Image swaps | Replace images with alternatives |
| Element hiding | Remove elements from the page |
| Style changes | Modify colors, fonts, sizing, spacing |
| Element rearranging | Move elements to different positions |
| HTML insertion | Add new elements to the page |
| Redirect tests | Send visitors to entirely different page URLs |
For changes beyond what the visual editor supports, you can write custom JavaScript and CSS in the code editor.
Experiment types
Section titled “Experiment types”A/B tests
Section titled “A/B tests”The most common experiment type. You create one or more variations of a page element and split traffic between them.
Example: Test two different headlines on a landing page.
- Control (A): “Start Your Free Trial”
- Variation (B): “See It In Action — Free for 14 Days”
Traffic is split 50/50. After reaching statistical significance, the winning headline is identified.
Multivariate tests (MVT)
Section titled “Multivariate tests (MVT)”Test multiple elements simultaneously to find the optimal combination.
Example: Test headline AND button color together.
- Headline A + Green button
- Headline A + Blue button
- Headline B + Green button
- Headline B + Blue button
MVT requires more traffic than A/B tests because it tests all combinations. Use MVT when you believe elements interact with each other.
Redirect tests
Section titled “Redirect tests”Send visitors to entirely different page URLs to test completely different page designs.
Example: Test a long-form landing page vs. a minimal page.
- Control:
/pricing(current detailed page) - Variation:
/pricing-v2(new minimal design)
Multi-page experiments
Section titled “Multi-page experiments”Test changes across multiple pages in a visitor’s journey. The variation is consistent across all pages in the experiment.
Example: Test a new navigation design across the entire checkout flow (cart → shipping → payment → confirmation).
Audiences and targeting
Section titled “Audiences and targeting”You control who sees experiments using audience conditions:
| Condition type | Examples |
|---|---|
| URL targeting | Specific pages, URL patterns, query parameters |
| Behavioral | New vs returning, pages visited, time on site |
| Geographic | Country, region, city |
| Technology | Browser, device type, operating system |
| Custom attributes | Data layer values, cookies, JavaScript variables |
| Third-party | DMP segments, Google Analytics audiences |
Audiences can be combined with AND/OR logic. For example: “Returning visitors from the US on mobile devices who have visited the pricing page.”
Metrics and goals
Section titled “Metrics and goals”Every experiment needs metrics to measure success:
| Metric type | What it measures | Example |
|---|---|---|
| Click | Clicks on a specific element | ”Sign Up” button clicks |
| Pageview | Visits to a specific page | Visits to /thank-you |
| Custom event | JavaScript-triggered events | Form submissions, video plays |
| Revenue | Purchase value | Cart value at checkout |
| Engagement | Time on page, scroll depth | Scroll to bottom of page |
Set a primary metric (the main thing you are optimizing for) and secondary metrics (additional insights). Optimizely adjusts for multiple comparisons to prevent false positives.
Stats Engine
Section titled “Stats Engine”Web Experimentation uses Stats Engine, Optimizely’s statistical methodology that provides always-valid results.
How it differs from traditional A/B testing:
| Traditional testing | Stats Engine |
|---|---|
| Fixed sample size required upfront | Check results any time |
| Can only evaluate at the end | Always-valid confidence intervals |
| p-value can be misleading if checked early | Results are accurate at every check |
| Manual correction for multiple metrics | Automatic false discovery rate control |
Stats Engine means you do not need to pre-calculate sample sizes or resist the urge to peek at results. The confidence intervals it reports are valid regardless of when you look.
When to use Web Experimentation
Section titled “When to use Web Experimentation”Use Web Experimentation when:
- You want to test visual changes (copy, images, layout)
- Marketers need to run tests without developer involvement
- You need results from the client side (browser-level changes)
- You want to run quick tests before a campaign launch
- You need personalization campaigns alongside experiments
- Your changes are on public web pages (not behind authentication)
For backend logic, API-level changes, or cross-platform experiments, use Feature Experimentation instead.