Skip to content
Optimizely CertificationOptimizely Certification Optimizely Certification

Learning Path: Experimentation Practitioner

⏱ 4-5 hours beginner
📜Foundationexperimentation

Who this path is for

Section titled “Who this path is for”

You are a marketer or developer who wants to start running experiments with Optimizely. You may be new to A/B testing entirely, or experienced with testing but new to the Optimizely platform. By the end of this path, you will understand how experimentation works, have run your first experiments, and know how to use feature flags for controlled rollouts.

How to use this path

Section titled “How to use this path”

Work through the steps in order. The first three steps build your conceptual foundation; steps 4 and 5 give you hands-on experience; steps 6 through 8 deepen your skills with guides and reference material.

Estimated total time: 4-5 hours (spread across multiple sessions is fine).

Step 1: Understand what experimentation is and why it matters

Section titled “Step 1: Understand what experimentation is and why it matters”

📖 Read — What is Optimizely Experimentation?

You will learn:

  • What problems experimentation solves for digital teams
  • How statistical testing removes guesswork from decisions
  • The difference between Web Experimentation and Feature Experimentation
  • Where experimentation fits in the Optimizely One platform

Time: 15 minutes

Step 2: Learn how Web Experimentation works

Section titled “Step 2: Learn how Web Experimentation works”

📖 Read — How Web Experimentation Works

You will learn:

  • How the Optimizely snippet delivers variations to visitors
  • The visual editor and how it modifies page elements
  • Audience targeting and traffic allocation
  • How results are calculated using Stats Engine

Time: 20 minutes

Step 3: Learn how Feature Experimentation works

Section titled “Step 3: Learn how Feature Experimentation works”

📖 Read — How Feature Experimentation Works

You will learn:

  • How feature flags decouple deployment from release
  • The SDK architecture and how flag decisions are evaluated
  • Environments, flag rules, and rollout strategies
  • How feature experiments differ from web experiments

Time: 20 minutes

Step 4: Run your first A/B test

Section titled “Step 4: Run your first A/B test”

🛠️ Do — Run Your First A/B Test

You will build:

  • A Web Experimentation project with the snippet installed
  • An A/B test with two variations using the visual editor
  • Audience targeting rules for your experiment
  • Goals and metrics to measure success
  • A running experiment with real traffic allocation

Time: 60 minutes

Step 5: Implement feature flags

Section titled “Step 5: Implement feature flags”

🛠️ Do — Implement Feature Flags

You will build:

  • A Feature Experimentation project with SDK integration
  • A feature flag with boolean and variable configurations
  • A targeted rollout using flag rules
  • A feature experiment comparing two flag variations
  • Code that evaluates flag decisions in your application

Time: 60 minutes

Step 6: Design effective experiments

Section titled “Step 6: Design effective experiments”

📋 Follow — Design an Experiment

You will learn:

  • How to form a strong hypothesis before building
  • Choosing primary and secondary metrics
  • Sample size planning and how long to run tests
  • Mutual exclusion groups for concurrent experiments
  • Common mistakes that invalidate results

Time: 25 minutes

Step 7: Analyze and act on results

Section titled “Step 7: Analyze and act on results”

📋 Follow — Analyze Experiment Results

You will learn:

  • How to read the results page and Stats Engine output
  • Statistical significance and what it means for decisions
  • Segmenting results by audience attributes
  • When to stop an experiment early
  • How to document and share experiment learnings

Time: 25 minutes

Step 8: Explore the experimentation reference

Section titled “Step 8: Explore the experimentation reference”

🔍 Reference — Browse these pages as needed:

Time: Browse as needed (30-60 minutes for initial read)

What to do next

Section titled “What to do next”

After completing this path, you are ready to:

  • Run production experiments — Apply hypothesis-driven testing to real business questions
  • Build a testing program — Establish experimentation culture and governance across your team
  • Explore personalization — Learn how experimentation connects to Visitor Groups and Personalization
  • Integrate with ODP — Use behavioral data from ODP to target experiments to the right audiences
  • Advance to platform architecture — Follow the Platform Architect learning path to understand how experimentation fits into the full Optimizely One stack
  • Prepare for certification — Visit Optimizely Academy for certification tracks