Feature Experimentation
Why feature flags change everything
Section titled “Why feature flags change everything”Traditional software releases are binary — a feature is either deployed to everyone or deployed to no one. This creates a coupling between deploying code and releasing features that introduces risk. A single broken feature in a deploy means rolling back everything.
Feature flags decouple deployment from release. You deploy code continuously, but features are hidden behind flags that you control remotely. Turn a flag on for 1% of users, monitor metrics, then gradually increase to 100%. If something breaks, turn off the flag instantly — no code rollback needed.
Feature Experimentation builds on this foundation by adding measurement. You do not just release features — you experiment with them, measuring which variations drive the best outcomes.
How Feature Experimentation works
Section titled “How Feature Experimentation works”The SDK model
Section titled “The SDK model”Feature Experimentation operates through SDKs embedded in your application code. When your code needs to decide whether to show a feature, it calls the SDK, which evaluates targeting rules and returns a decision.
using OptimizelySDK;
var optimizely = OptimizelyFactory.NewDefaultInstance(sdkKey);
var user = optimizely.CreateUserContext("user-123");
// Check if feature is enabled for this user
var decision = user.Decide("new_checkout_flow");
if (decision.Enabled)
{
// Show new checkout
var buttonColor = decision.Variables["button_color"];
RenderNewCheckout(buttonColor);
}
else
{
// Show existing checkout
RenderCurrentCheckout();
}import { createInstance } from '@optimizely/optimizely-sdk';
const optimizely = createInstance({ sdkKey: 'YOUR_SDK_KEY' });
await optimizely.onReady();
const user = optimizely.createUserContext('user-123', {
plan: 'enterprise',
country: 'US',
});
const decision = user.decide('new_checkout_flow');
if (decision.enabled) {
const buttonColor = decision.variables.button_color;
renderNewCheckout(buttonColor);
} else {
renderCurrentCheckout();
}from optimizely import optimizely
client = optimizely.Optimizely(sdk_key='YOUR_SDK_KEY')
user = client.create_user_context('user-123', {
'plan': 'enterprise',
'country': 'US',
})
decision = user.decide('new_checkout_flow')
if decision.enabled:
button_color = decision.variables['button_color']
render_new_checkout(button_color)
else:
render_current_checkout()Feature flags
Section titled “Feature flags”A feature flag is a named toggle with optional variables. It has three components:
| Component | What it does | Example |
|---|---|---|
| Key | Unique identifier | new_checkout_flow |
| Variations | Different configurations | control (old) vs treatment (new) |
| Variables | Configurable values per variation | button_color: "green", show_trust_badges: true |
Variables are the key to moving beyond simple on/off toggles. Instead of just enabling a feature, you can configure how it behaves — what color, what copy, what algorithm — and vary those configurations across experiment variations.
Targeting and audiences
Section titled “Targeting and audiences”You control who sees a feature using audiences defined by user attributes:
// Pass user attributes when creating context
const user = optimizely.createUserContext('user-456', {
plan: 'enterprise',
country: 'DE',
employee_count: 500,
signed_up: '2024-01-15',
});
// The SDK evaluates targeting rules server-side
// and returns a decision based on these attributes
const decision = user.decide('advanced_analytics');Audience rules are defined in the Optimizely UI (not in code). This means product managers can change targeting without a deploy — for example, expanding a feature from “enterprise US customers” to “all enterprise customers” is a UI change, not a code change.
Progressive rollouts
Section titled “Progressive rollouts”Rollouts let you release a feature gradually:
- 1% traffic — Smoke test with a small population
- 10% traffic — Monitor error rates and performance
- 50% traffic — Validate at scale
- 100% traffic — Full release
At each stage, you can monitor metrics and halt the rollout if problems appear. The SDK ensures consistent bucketing — a user who was in the 1% group stays in it at 10%, 50%, and 100%.
Experiments on features
Section titled “Experiments on features”Running an experiment on a feature flag adds measurement. Instead of just rolling out and hoping for the best, you split traffic between variations and measure which one performs better.
Experiment setup:
- Define the feature flag with variations
- Create an experiment targeting the flag
- Set primary and secondary metrics
- Allocate traffic (e.g., 50/50 between control and treatment)
- Launch and wait for statistical significance
Mutual exclusion groups
Section titled “Mutual exclusion groups”When running multiple experiments simultaneously, you need to ensure they do not interfere with each other. Mutual exclusion groups guarantee that a user is only in one experiment from the group at a time.
SDK architecture
Section titled “SDK architecture”The SDK is designed for performance and reliability:
| Aspect | How it works |
|---|---|
| Initialization | SDK downloads a datafile (JSON configuration) at startup |
| Decision making | All decisions are made locally — no network call per decision |
| Datafile updates | Background polling or webhook-triggered updates |
| Offline fallback | If the CDN is unreachable, the SDK uses the last cached datafile |
| Event tracking | Events are batched and sent asynchronously |
This architecture means feature flag checks add sub-millisecond latency to your application. The SDK does not call home for every decision.
Available SDKs
Section titled “Available SDKs”| Platform | Package | Use case |
|---|---|---|
| JavaScript (Browser) | @optimizely/optimizely-sdk | Client-side web apps |
| JavaScript (Node.js) | @optimizely/optimizely-sdk | Server-side Node apps |
| React | @optimizely/react-sdk | React components with hooks |
| C# (.NET) | Optimizely.SDK | .NET applications |
| Python | optimizely-sdk | Python backends |
| Java | com.optimizely.ab:core-api | Java applications |
| Go | github.com/optimizely/go-sdk | Go services |
| Ruby | optimizely-sdk | Ruby applications |
| PHP | optimizely/php-sdk | PHP applications |
| Swift | OptimizelySwift | iOS apps |
| Android | com.optimizely.ab:android-sdk | Android apps |
When to use Feature Experimentation
Section titled “When to use Feature Experimentation”Use Feature Experimentation when:
- You need server-side control over feature visibility
- You want to reduce deployment risk with progressive rollouts
- Your experiment involves backend logic (algorithms, pricing, APIs)
- You need cross-platform consistency (web + mobile + API)
- You want instant kill switches for new features
- Your team follows trunk-based development with continuous deployment