Skip to content
Optimizely CertificationOptimizely Certification Optimizely Certification

Analytics-Driven Experimentation

⏱ 40 minutes intermediate
📜CoreAnalyticsexperimentationODP

Why experiments should start with data, not opinions

Section titled “Why experiments should start with data, not opinions”

Most experimentation programs fail because they test ideas generated in brainstorming meetings rather than problems identified in data. A team might spend weeks testing button colors while a 40% drop-off on the pricing page goes unaddressed.

Analytics tells you where visitors struggle. Experimentation tells you which solution works. ODP enriches both with customer-level behavioral context. This recipe connects the three into a cycle: find the problem in Analytics, hypothesize a fix, test it with Experimentation, and verify the result back in Analytics.

Architecture overview

Section titled “Architecture overview”
┌────────────────────────────────────────────────┐
│         Analytics: Identify Problem             │
│  Funnel analysis reveals 45% drop-off           │
│  on pricing page for returning visitors         │
└───────────────────────┬────────────────────────┘
                        │ insight
                        ▼
┌────────────────────────────────────────────────┐
│         ODP: Enrich with Behavior Data          │
│  Returning visitors who drop off have           │
│  viewed pricing 3+ times without converting     │
└───────────────────────┬────────────────────────┘
                        │ refined hypothesis
                        ▼
┌────────────────────────────────────────────────┐
│      Experimentation: Test the Hypothesis       │
│  A/B test: Control vs simplified pricing        │
│  table with comparison toggle                   │
└───────────────────────┬────────────────────────┘
                        │ results
                        ▼
┌────────────────────────────────────────────────┐
│        Analytics: Validate Impact               │
│  Monitor pricing page conversion rate           │
│  for 30 days post-deployment                    │
└────────────────────────────────────────────────┘

Step 1: Find the conversion bottleneck

Section titled “Step 1: Find the conversion bottleneck”

Open Analytics and review your key conversion funnels. Look for steps with disproportionate drop-off rates. Focus on pages where:

  • Drop-off exceeds 30% between funnel steps
  • The page has high traffic but low conversion
  • Bounce rate increased recently without explanation

Document the specific metric, the page, and the audience segment affected.

Step 2: Enrich with ODP behavioral data

Section titled “Step 2: Enrich with ODP behavioral data”

Query ODP to understand who drops off and why. Filter customer profiles by the behavior pattern (visited pricing page, did not convert) and examine:

  • How many times they visited before dropping off
  • What other pages they viewed in the same session
  • Whether they match a specific ODP segment (enterprise vs SMB, new vs returning)

This context turns a vague observation into a testable hypothesis.

Step 3: Form a hypothesis

Section titled “Step 3: Form a hypothesis”

Structure your hypothesis as: “For [audience segment], changing [page element] from [current state] to [proposed state] will increase [metric] by [expected amount] because [reason from data].”

Example: “For returning visitors who have viewed pricing 3+ times, simplifying the pricing table from a feature matrix to a plan comparison toggle will increase pricing-to-checkout conversion by 15% because the current table overwhelms users comparing plan details.”

Step 4: Build and launch the experiment

Section titled “Step 4: Build and launch the experiment”

Create the experiment in Web Experimentation:

  1. Target the audience identified in Step 2 using ODP segment criteria
  2. Build the variation that implements your proposed change
  3. Set the primary metric to the conversion event from Step 1
  4. Add secondary metrics for engagement (scroll depth, time on page, click-through)
  5. Calculate the required sample size and set the experiment duration

Step 5: Analyze and deploy

Section titled “Step 5: Analyze and deploy”

When the experiment reaches statistical significance:

  • If the variation wins, deploy the change and monitor in Analytics for 30 days to confirm sustained impact
  • If the control wins, revisit the ODP behavioral data to understand why the hypothesis was wrong
  • Document the result in your experimentation knowledge base for future reference

When to use this pattern

Section titled “When to use this pattern”

Use this data-driven cycle when your experimentation program generates more test ideas than capacity allows. The analytics-first approach ensures you test the highest-impact opportunities. Skip this process for low-risk design tweaks where the cost of testing exceeds the potential gain.