--- title: Local holdouts slug: feature-experimentation/docs/rF0O-local docTags: createdAt: 2026-07-31T15:14:36.421Z --- :::hint{type="success"} **Products** - Optimizely Feature Experimentation The availability of features may depend on your plan type. Reach out to your Customer Success Manager if you have any questions. ::: Local holdouts extend Optimizely [global holdout](docId\:DogKLoT6bPNNCcoA2ch7a) capabilities. A global holdout applies to one or more entire projects. A local holdout applies only to the specific rules you choose. This lets you hold traffic back from a targeted set of experiments instead of the whole project. Local holdouts and global holdouts work independently. Creating a local holdout does not affect any existing global holdouts. A holdout keeps a group of visitors out of your experiments. This gives you an untouched control group to measure the cumulative impact of an experimentation program. If you are new to holdouts, review the [core concepts of Feature Experimentation](docId:-yq2-thNRv7-SgwCXpG90) first. A global holdout applies to every rule in the project or projects you select. Local holdouts add precision by applying only to the rules you choose. Use them to: - Measure the combined impact of a specific initiative, such as a set of checkout experiments, without holding traffic back from unrelated rules. - Keep the rest of the project running at full traffic while you measure the rules in the holdout. Local holdouts keep the same statistical rigor and automated reporting as global holdouts, while giving you control over the holdout's boundaries. ## How local holdouts work When you start a local holdout, Optimizely holds back a defined percentage of eligible visitors from the rules you selected. Held-back visitors receive the baseline experience (the default "off" [flag variation](docId\:m5k7AZmkz0gSS20cE3rRi)) for those rules, regardless of what is running. Optimizely buckets everyone else into the rules as normal. Optimizely aggregates results across the selected rules. It compares the holdout group against visitors who saw those rules to give you the cumulative impact of that set of rules. Local holdouts have the following key behaviors: - **Scope is fixed at start** – The set of rules in a holdout is locked when the holdout starts. This keeps how long each rule runs consistent, so results stay statistically valid. - **All-or-nothing per rule** – A rule is either fully in or fully out of a holdout. You cannot include only certain variations of a rule. - **Consistent hold-back** – A held-back visitor receives the baseline experience for every rule in the holdout, not only some of them. ## Permissions You can assign roles per project. See [Manage collaborators in Feature Experimentation](docId\:bNAxj4srKHVczBp27i1DH). - **Create or edit a local holdout** – Editor, Collaborator, and Admin. - **View holdout results** – All roles (including Viewer). - **Create audiences from a holdout** – Editor, Collaborator, and Admin. ## Create a local holdout 1. Go to **Flags > Holdouts**. 2. Click **Create Holdout**. 3. Enter a unique **Name**. 4. Edit the **Holdout Key** and optionally add a **Description**. Valid keys contain alphanumeric characters, hyphens, and underscores, are limited to 64 characters, and cannot contain spaces. You cannot modify the key after you create the holdout. ::Image[]{src="https://api.archbee.com/api/optimize/BrbvVORKJ5iIeSMZye5kI/5LmDEvNi-13Qeza_XZKcg_create-holdout-create-holdout-name-key-description-fields.png" size="60" isUploading="false" width="1286" height="571" darkWidth="1286" darkHeight="571" showCaption="false"} 5. Under **Scope**, select the **Environment** this holdout applies to. See [Manage environments in Feature Experimentation](docId\:YpSOMYlWkUccLCL-uB-ZE). 6. Select **Local holdout** (To hold traffic back from entire projects instead, select **Global holdout**). 7. Add the rules to include. Rules include [A/B tests](docId:2duP0GL8keJuYvAoQChSU) and other flag rules in the project. A local holdout must include at least one rule. ::Image[]{src="https://api.archbee.com/api/optimize/BrbvVORKJ5iIeSMZye5kI/Ka9Rvmn6eePsxn-_xdwgr_scope-local-holdout-scope-rule-selection.png" size="60" isUploading="false" width="1285" height="615" darkWidth="1285" darkHeight="615" showCaption="false"} 8. Under **Metrics**, click **Add Metric** and add one or more metrics. Metrics apply across all rules in the holdout, and the selected environment determines which metrics are available. See [Choose metrics in Feature Experimentation](docId\:e2qGf29S6pO8SW90_IVuH). 9. Under **Traffic Allocation**, enter the percentage of visitors to withhold from each rule in the holdout. Optimizely recommends 1–5% and shows a warning if you exceed 5%. 10. (Optional) Under **Audiences**, search for and add audiences to apply the holdout to specific visitors. The default audience is **Everyone**. See [Target audiences in Feature Experimentation](docId\:UDBvr5a8WHNkyOqzPhOr9). 11. Click **Create Holdout**. ::Image[]{src="https://api.archbee.com/api/optimize/BrbvVORKJ5iIeSMZye5kI/mlKl_XDFHt1uK6iCf-Tib_metrics-metrics-traffic-allocation-and-audiences.png" size="60" isUploading="false" width="1285" height="1127" darkWidth="1285" darkHeight="1127" showCaption="false"} Audience conditions set on an individual experiment take priority over audience conditions set on the holdout. For example, an audience excluded from an experiment stays excluded from it, even if the holdout does not exclude that audience. Optimizely creates the holdout in a draft state. It does not affect traffic until you start it. See [Start a holdout](docId\:rF0OPLtVhEbE4NEcu2gcG). ## Tag and target holdout users Local holdouts tag every user assigned to the holdout so you can reuse that group of users after the holdout ends: - **Automatic tagging** – Optimizely tags each visitor in the holdout by an attribute or ID. - **Audience creation** – Create an [audience](docId\:UDBvr5a8WHNkyOqzPhOr9) to **Exclude holdout users** or **Target holdout users** for follow-up experiments and analysis. - **Export** – Export the list of holdout visitors for further analysis. ## View holdout reports You can open a holdout to see its results dashboard, which reports the cumulative impact of the selected experiments against the holdout group. The dashboard follows the same conventions as the [Optimizely Experiment Results page](docId\:mrUblIuNVOZc1dvjWdsbw). To read statistical significance and confidence intervals, see [Analyze results in Feature Experimentation](docId:_K6n7h_MZ8b0xn6yWPrrq). The dashboard can show: - **Cumulative impact** of the holdout group on your primary metrics. - **Per-experiment contribution** to the overall impact. - **Trend over time** for the holdout. - **Comparison** against other holdouts or historical data. To break impact down by user attribute, [segment your results](docId\:E4kGQiwGcruXWHm-1n_HW). Export results (including an executive summary) to PDF or CSV. ## Start a holdout After creating the holdout, it displays in the **Holdouts** tab, where you can start it. Click **More (...)** for the holdout and click **Start**. ![](https://api.archbee.com/api/optimize/BrbvVORKJ5iIeSMZye5kI/Yn68cfAdXqaqdxDfT5wnC_flags-holdouts-holdouts-list-start-draft-holdout.png) Visitors assigned to this holdout only see the default "off" variation for any flags within the holdout, regardless of any experiments they might otherwise be included in by Feature Experimentation. When you start a holdout, you cannot pause it; only permanently conclude it. To see holdouts that Feature Experimentation applied to a rule, go to **Flags > Environment > Rule**. The holdouts applied, along with the traffic allocated, display. ![](https://api.archbee.com/api/optimize/BrbvVORKJ5iIeSMZye5kI/DszvOyJN0d5jvmFOwputK_landing-page-landing-page-flag-ruleset-holdouts-panel.png) ## Conclude or delete a holdout The available actions depend on whether you have started the holdout: - **Holdout started** – You cannot delete a holdout that has started. Click **More (...) > Conclude** instead. The holdout status changes to Concluded. Every user in the holdout is rebucketed and begins receiving variations from the running A/B Tests, Targeted Deliveries, and Multi-Armed Bandits. You cannot re-enable a concluded holdout. - **Holdout not started** – Click **More (...) > Delete** to remove the holdout permanently from the project. The action cannot be undone. After a holdout concludes, click **More (...) > Archive** when it is no longer relevant. Archives preserve the historical results and remove the holdout from the active list. To unarchive a holdout, click **More (...) > Unarchive.** The deletion safeguard protects long-running holdout data from accidental loss. For example, a holdout that ran for a quarter cannot be deleted, so its results remain available for review and reporting. ## Local holdouts versus global holdouts | Capability | Global holdout | Local holdout | | -------------------- | ------------------------------------------- | ---------------------------------------------------------- | | Applies to | One or more entire projects (all rules) | Specific rules you select | | Rule-level selection | No (includes every rule) | Yes | | Minimum to configure | At least one project | At least one rule | | Best for | Program-wide baseline across whole projects | Measuring a specific set of rules without affecting others | A visitor cannot be in both a global and a local holdout at the same time. If both apply, the global holdout takes precedence. If you need to scope a holdout to specific rules, use a local holdout. If a visitor qualifies for multiple local holdouts, the holdout that is highest in your configured priority order wins. You manage this order in the holdout settings. This is similar to how [mutually exclusive experiments](docId\:Jx601K4Y1NWOlTXhKMZNy) prevent a visitor from entering overlapping tests. ## Frequently asked questions :::ExpandableHeading ### Can I add rules to a holdout after it has started? No. Adding rules mid-flight introduces different exposure durations, which compromises statistical validity. Scope is locked when the holdout starts. ::: :::ExpandableHeading ### What happens if a user qualifies for multiple local holdouts? The holdout highest in your configured priority order wins. You set this order in the holdout settings. ::: :::ExpandableHeading ### Can I run a local holdout and a global holdout at the same time? A single user cannot be in both. Global holdouts take precedence. If you need to scope a holdout to specific rules, use a local holdout :::