Mutually exclusive experiments in Feature Experimentation

In Optimizely Feature Experimentation, experiments may overlap, exposing a single user to multiple experiments simultaneously. Mutually exclusive experiments prevent this by ensuring a user is exposed to only one particular experiment at a time. For experiments that are not mutually exclusive, Feature Experimentation uses a unique, random value for each experiment to determine user entry. This value is not mutually exclusive across experiments, which means users may enter multiple experiments. For more information, see How bucketing works.

For example, there are two experiments: Experiment A and Experiment B. Each experiment receives 20% traffic allocation (the percentage of total traffic that is eligible for the experiment). The traffic allocation would be the following:

In the example above, Experiment A's and Experiment B's results may be skewed. If users who see both A and B behave differently from those who see just A or B, then the results for A and B are skewed by the overlap. This is called an interaction effect.

If Experiment A and Experiment B are mutually exclusive, Feature Experimentation chooses the same random value (unique to the exclusion group) to bucket users in experiments A and B. This method ensures that experiments cannot overlap for the same users. If experiments A and B are mutually exclusive, the traffic allocation looks something like the following:

Feature Experimentation also ensures mutual exclusivity between experiments in an exclusion group that run at different times.

Before implementing exclusion groups and mutual exclusion, keep the following best practices in mind:

In Feature Experimentation projects, you can make two or more experiments mutually exclusive of each other using exclusion groups. Using exclusion groups does not require any change in the SDKs. The Decide and Track Events methods automatically evaluate whether an experiment is in a group and assign users to the appropriate experiment and variation.

If you want to implement exclusion groups using the Feature Experimentation APIs instead, see Use mutual exclusion in the developer documentation.

To create an exclusion group in the UI, complete the following:

To add an experiment to an exclusion group before running your experiments, complete the following:

Remove finished experiments from the exclusion group to make space for other experiments.

To archive an exclusion group:

To find archived exclusion groups and unarchive them:

The following table shows how exclusion groups can interact with other bucketing methods.