Run Feature Rollouts in Feature Experimentation
Products
- Optimizely Feature Experimentation
The availability of features may depend on your plan type. Reach out to your Customer Success Manager if you have any questions.
This feature is currently in beta. Contact your Customer Success Manager or sign up for the beta on Optimizely.com.
Feature Rollouts are a rule type in Optimizely Feature Experimentation that combine the simplicity of Targeted Deliveries with the measurement power of A/B tests. Use Feature Rollouts to release a single variation to production progressively while you track impressions, conversions, and business metrics.
Feature Rollouts give you full impact analytics, metric tracking, and flag evaluation insights for feature releases that are not designed as experiments. A typical use case: an A/B test concludes with a winning variation, and you want to release that winner gradually while you monitor its effect on key business metrics. Rather than create a single-variation A/B test as a workaround, you create a Feature Rollout. The rule type provides measurement, allowlists, evaluation-order control, and impression tracking.
When to use Feature Rollouts
Use Feature Rollouts when you:
- Release a winning variation from a concluded experiment to production traffic.
- Roll out a feature progressively (10%, 25%, 50%, then 100%) and measure impact at each stage.
- Release a feature to a defined audience, such as beta users or a region, and track performance.
- Pre-test a release with an internal allowlist before broader traffic exposure.
Configuration overview
To configure a Feature Rollout, complete the following:
- (Prerequisite) Create a flagCreate a flag in your Feature Experimentation project.
- (Prerequisite) Handle user IDsHandle user IDs.
You should configure a user profile serviceuser profile service to ensure consistent user bucketing if you are using a server-side SDK.
- If you have not done so yet, implement the Optimizely Feature Experimentation SDK's Decide method in your application's codebase through a flagflag.
- Test your Feature Rollout rule in a development environment. See Test and troubleshootTest and troubleshoot.
- Discard any test user events and enable your Feature Rollout rule in a production environment.
Compare Feature Rollouts, Targeted Deliveries, and A/B tests
The following table compares the three rule types across capabilities that matter when you choose between them:
Capability | Targeted Delivery | Feature Rollout | A/B test |
|---|---|---|---|
Number of variations | One | One | Two or more |
Impression tracking | No | Yes | Yes |
Metrics and analytics | No | Yes | Yes |
Allowlist for pre-testing | No | Yes | Yes |
Configurable evaluation order | No, always last | Yes | Yes |
Statistical analysis | Not applicable | Single-variation results with confidence intervals | Multi-variation comparison with significance testing |
Consumes monthly active users (MAU) and impressions | No | Yes | Yes |
Best for | Simple toggles where measurement is not required | Progressive release of a chosen variation with measurement | Comparison of multiple variations to determine a winner |
Create a Feature Rollout
Create a Feature Rollout from a winning experiment variation or build one from scratch on a flag. Both paths produce the same rule type with the same configuration options.
- Go to Flags, select your flag, and select your environment.
- Click Add Rule and select Feature Rollout.
- Enter a Name for the rule.
- The Key field auto-populates from the Name. You can optionally update it.
- (Optional) Click Add description to add a description of the feature rollout.
- (Optional) Search for and add audiences. To create an audience, see Target audiencesTarget audiences. Audiences evaluate in the order in which you drag and drop them. You can choose whether to match each user on any or all of the audience conditions.
- Set the Traffic Allocation to assign a percentage of your audience to bucket into the experiment.
- Add Metrics based on tracked user events. See Manage events in Feature ExperimentationManage events in Feature Experimentation for information on how to create events using the UI or Create eventsCreate events for how to use the Feature Experimentation REST API.
- For information about selecting metrics, see Choose metrics in Feature ExperimentationChoose metrics in Feature Experimentation.
- For instructions on creating metrics, see Create a metric in Optimizely using the metric builderCreate a metric in Optimizely using the metric builder.
- Select the variation to deploy.
- Set the traffic allocation. For example, 10%, the recommended starting point.
- (Optional) Click Allowlist: Force up to 50 users into any variation(s) and enter the User ID. See AllowlistingAllowlisting.
- Click Save.
Click Run on the rule to start the Feature Rollout. If the ruleset (flag) is not running, click Run on it.
Permissions and roles
- Create or modify a Feature Rollout – requires the manage_production or manage_flags role.
- View rollouts and analytics – requires the view_flags role for read-only access.
Optimizely records all Feature Rollout create, read, update, and delete operations in change history for audit purposes.