Core concepts of Feature Experimentation

Optimizely Feature Experimentation provides a general interface for running experiments throughout your applications and services. Feature Experimentation is designed for maximum customization because teams typically run experiments in different parts of a stack with different requirements. You use projects, environments, rulesets, datafilesdatafiles, and clients to adapt one or more Feature Experimentation SDKs to your development workflow.

Feature flags are more than simple on or off toggles for exposing features. Add remote configuration variablesvariables to flag variationsvariations and build A/B testsA/B tests on top of feature flags. Use the decide methods to return a flag decision for a user. The response includes the flag's on or off status, variable values, variation key, and whether the flag dispatched a decision event.

Flag rules are how you enable and run flag deliveries and experiments. Feature Experimentation includes the following flag rules:

Use an A/B test when you need a controlled statistical comparison between variations with fixed traffic splits. A/B tests are better suited for decisions that require high statistical confidence and a clean separation between control and treatment groups.

Use multi-armed bandit optimization when you want to maximize conversions during the experiment itself. The algorithm reallocates traffic dynamically toward the best-performing variation, which means the experiment does not maintain fixed traffic splits. This reduces the statistical rigor of the comparison but increases performance during the experiment window. Multi-armed bandit optimization is typically a better fit for short-lived tests or situations where losing traffic to underperforming variations has a measurable cost.

Feature Experimentation assigns users to variations through a process called bucketing. The SDK uses the user ID as an input to a deterministic hash function, which produces a consistent, reproducible variation assignment. A user with the same ID always receives the same variation as long as the traffic distribution has not changed.

A Feature Experimentation project lets you isolate each of your teams or applications in a separate shared workspace. Projects are where a team builds experiments, manages feature flags, defines target audiences, and more. Each project has its own set of permissions.

Projects have a many-to-many relationship with your application stack. You can have many different projects to manage different parts of a single application or use a single project to manage omnichannel experiments across the website back end and mobile app.

Because your account already includes a project, creating another project is an optional step. See Manage projectsManage projects.

Use environmentsenvironments in Optimizely to develop and validate your experiment internally before deploying changes to your production website or application, whether you use a formal deployment environment or not. Each project can have one or more environments.

For example, it is common to configure both staging and production environments in Optimizely. You typically develop and test your experimentstest your experiments in staging. You can configure different permissionsdifferent permissions for who can make changes in each of these environments.

A ruleset is a collection of rules associated with a flag within a specific environment. For details on how Feature Experimentation qualifies users for rules and the order in which it executes them, see Interactions between flag rulesInteractions between flag rules.

Each environment in a project has a corresponding OptimizelyConfig, which is serialized as a datafiledatafile. This file captures the configuration data of all your experiments, such as feature flags and event tracking, in JSON. When you make changes in a project, Feature Experimentation automatically updates each environment's datafile. By syncing a local copy of this datafile, the SDK can run experiments without blocking network requests to an outside API, ensuring near-zero latency.

Datafile changes are also uploaded to the Optimizely content delivery networkcontent delivery network (CDN), cdn.optimizely.com, and this process generally takes a few minutes.

Feature Experimentation generates a different version of the datafile for each environment you configure on your account. This lets you toggle a feature flag in one environment while keeping it paused in another. You do not need to duplicate your code, your Optimizely project, or experiments within your project.

A client is an instance of the Optimizely Feature Experimentation SDK running with a specific datafile and other configuration settings. To use the SDK, get the appropriate datafile and then instantiate a client with it. This client exposes the methods you need to decide if a user sees a flag, track events, and perform other tasks.

The AndroidAndroid, FlutterFlutter, and SwiftSwift SDKs (also called the mobile SDKs) offer the additional abstraction of a client manager. The manager handles syncing datafiles and generating the associated clients for you, along with other optimizations around event dispatching and persisting user state on the device. Using the manager is optional, and you can instantiate a client directly if you prefer complete control.

An impression is a unit of measurement of flag usage. Optimizely Feature Experimentation counts an impression each time the SDK sends a decision event. See impressionsimpressions for information about the following:

See Impression in Optimizely ExperimentationImpression in Optimizely Experimentation for information about the differences between how impressions are counted in Optimizely Feature Experimentation in contrast to Optimizely Web ExperimentationOptimizely Web Experimentation.

You can use the decide methods to return a flag decision for a user. The flag decision includes the flag-enabled or disabled status and the flag variation.

A decision event is an SDK action that triggers a network request. To avoid the request, you can disable dispatching decision events for the decide method using the SDK's event dispatcher.

For information, see the following Feature Experimentation's SDK decide method documentation:

You can use the track eventtrack event method to record user events, such as clicks, page views, or purchases. Tracking these events is important for measuring the impact of your feature flags and experiments, as it lets you analyze user behavior and evaluate the performance of different variations. Using this method you can also include optional parameters, such as user attributes or event tags, to provide additional context.

For detailed guidance on using the Track method in specific SDKs, refer to the following documentation:

For Optimizely Agent, see Use Optimizely AgentUse Optimizely Agent.

See the following links for explanations of core concepts and reference topics: