Model: Holdout

1 min

Model Parameters

Holdout
Example
```
{
  "archived": false,
  "audience_conditions": [
    "or",
    {
      "audience_id": 5340810295
    }
  ],
  "created_by_user_email": "[email protected]",
  "created_time": "2023-12-15T10:00:00.000Z",
  "description": "Holdout group to measure the overall impact of our growth experiments throughout the year",
  "environment_key": "production",
  "holdout_results_api_token": "SampleToken.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9",
  "id": 12345,
  "key": "annual_growth_holdout",
  "metrics": [
    {
      "aggregator": "unique",
      "display_title": "Conversion Rate",
      "scope": "visitor",
      "winning_direction": "increasing"
    }
  ],
  "name": "Annual Growth Control Group",
  "project_id": 98765,
  "revision": 3,
  "scope": "global",
  "start_time": "2024-01-01T00:00:00.000Z",
  "status": "running",
  "traffic_allocation": 1000,
  "updated_time": "2024-01-01T10:00:00.000Z",
  "urn": "holdout.flag.optimizely.com::12345"
}
```
- archivedboolean
  Description
  Whether the holdout is archived. Archived holdouts are hidden from normal views but retain their historical data and configuration.
- audience_conditionsobject[]
  Example
  ["["or",{"audience_id":5340810295},{"audience_id":1234567890}]"]
  Description
  Defines which audiences this holdout applies to. If omitted, the holdout applies to all users. Uses logical operators like 'and', 'or', 'not' to combine multiple audience conditions.
- audience_idsinteger[]
  Example
```
[
  5340810295,
  1234567890
]
```
  Description
  Warning: Deprecated
  DEPRECATED: Use audience_conditions instead. List of audience IDs that this holdout applies to. This field is maintained for backward compatibility.
- created_by_user_emailstring
  Example
  [email protected]
  Description
  Email address of the user who created the holdout.
- created_timestring<date-time>
  Example
  2023-12-15T10:00:00.000Z
  Description
  Timestamp when the holdout was originally created.
  Format
  date-time
- descriptionstring
  Example
  Holdout group to measure the overall impact of our growth experiments throughout the year
  Description
  Optional detailed description explaining the purpose and context of this holdout. Helps team members understand what this holdout is measuring.
  Max Length
  1000
- end_timestring<date-time>
  Example
  2024-12-31T23:59:59.000Z
  Description
  Timestamp when the holdout finished or will finish. Null for holdouts that are still running or have no defined end time.
  Format
  date-time
- environment_keystring
  Example
  production
  Description
  The environment where this holdout is active (e.g., 'production', 'staging'). The holdout only affects traffic in this specific environment.
- environment_namestring
  Example
  Production
  Description
  The human-readable name of the environment where this holdout is active.
- exclude_targeted_deliveriesboolean
  Description
  When true, Targeted Delivery rules are excluded from this holdout, so holdout users are not withheld from them. Targeted Deliveries represent permanent ship decisions rather than active experiments, so there is nothing to measure by withholding them. Applies to global holdouts only; ignored for local holdouts. Can only be changed while the holdout is in draft status. Defaults to false.
  Default
  false
- holdout_results_api_tokenstring
  Example
  SampleToken.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9
  Description
  API token for accessing holdout results and analytics data. This token is dynamically generated for each API response and provides secure access to holdout-specific results endpoints.
- idintegerrequired
  Example
  12345
  Description
  System-generated unique identifier for the holdout.
- keystringrequired
  Example
  annual_growth_holdout
  Description
  Unique user-defined identifier for the holdout within the project. Must contain only alphanumeric characters, underscores, and hyphens.
  Max Length
  64
  Pattern
  ^[a-zA-Z0-9_\-]+$
- metricsMetric[]
  Description
  List of metrics being tracked and analyzed for this holdout. These metrics help measure the impact of experiments on the control group.
  - aggregatorstring<unique | count | sum | ratio>required
    Description
    The aggregation function for the numerator of the metric. 'unique' measures the number of unique visitors/sessions that include the specified Event. 'count' measures the total number of occurrences of Event for the scope (visitor/session). 'sum' is the sum of the 'field' value
    Allowed
    uniquecountsumratio
  - display_titlestring
    Description
    An optional title for the unique metric configuration. If not provided, the event name will be used.
  - display_unitstring
    Description
    An optional name for the unit of the unique metric configuration.
  - event_idinteger<int64>
    Example
    32334454244
    Description
    The ID for the Event to select data from. Omitted for global metrics that are not relative to a specific Event, i.e. "overall revenue"
    Format
    int64
  - event_propertiesobject
    - filterobject
      - combine_operatorstring<or | and>
        Allowed
        orand
      - conditionsEventPropertyCondition[]
        namestringrequired
        operatorstring<equal_to | not_equal_to | greater_than | greater_than_or_equal_to | less_than | less_than_or_equal_to | contains | does_not_contain>required
        Allowed
        equal_tonot_equal_togreater_thangreater_than_or_equal_toless_thanless_than_or_equal_tocontainsdoes_not_contain
        typestring<string | number | boolean>required
        Allowed
        stringnumberboolean
        valuestringrequired
  - event_typestring
  - fieldstring<revenue | value>
    Description
    The field to aggregate for the numerator of the metric. Required when 'aggregator' = 'sum', otherwise omitted
    Allowed
    revenuevalue
  - idstring
    Description
    A unique identifier for the metric configuration.
  - metricsSubMetric[]
    - aggregatorstring
    - display_titlestring
    - event_idinteger
    - event_typestring
    - fieldstring
    - rolestring<numerator | denominator>
      Allowed
      numeratordenominator
    - scopestring
  - project_idinteger
    Description
    A unique identifier for the project associated with the metric.
  - scopestring<session | visitor | event>
    Description
    Specifies how Events should be grouped together. Can also be thought of as the denonimator of the metric. 'session' divides by the number of sessions. "Influenced sessions", or sessions that do not contain a decision Event but carry a decision from a previous session are not included in counts for numerator or denominator. 'visitor' divides by the number of visitors. 'event' divides by the total occurrences (impressions) of the specified Event
    Allowed
    sessionvisitorevent
  - visibilitystring<account | experiment | project>
    Description
    The visibility of the metric.
    Allowed
    accountexperimentproject
  - winning_directionstring<increasing | decreasing>required
    Description
    The winning direction of this metric
    Allowed
    increasingdecreasing
- namestringrequired
  Example
  Annual Growth Control Group
  Description
  Human-readable name for the holdout displayed in the UI.
  Max Length
  255
- project_idinteger<int64>
  Example
  98765
  Description
  The unique identifier of the project this holdout belongs to.
  Format
  int64
- projectsobject[]
  Example
```
[
  {
    "id": 123
  }
]
```
  Description
  Project objects returned by cross-project GET endpoints. For global-scope holdouts, lists target project IDs. Empty array for local-scope holdouts. Frontend resolves project names via entity-management API.
  - idinteger<int64>
    Description
    Project ID.
    Format
    int64
- revisioninteger
  Example
  3
  Description
  Revision number that increments with each change to the holdout. Used for tracking modifications and ensuring data consistency.
- rulesobject[]
  Example
```
[
  {
    "flag_key": "my_flag",
    "id": 456,
    "key": "experiment_rule_a",
    "name": "Experiment Rule A",
    "project_id": 123
  }
]
```
  Description
  Enriched rule objects returned by cross-project GET endpoints. For local-scope holdouts, lists target rules with metadata. Empty array for global-scope holdouts.
  - flag_idinteger<int64>
    Description
    ID of the flag containing this rule.
    Format
    int64
  - flag_keystring
    Description
    Key of the flag containing this rule.
  - flag_namestring
    Description
    Name of the flag containing this rule.
  - idinteger<int64>
    Description
    Rule ID.
    Format
    int64
  - keystring
    Description
    Rule key.
  - namestring
    Description
    Rule name.
  - project_idinteger<int64>
    Description
    ID of the project this rule belongs to.
    Format
    int64
- scopestring<global | local>
  Example
  global
  Description
  Defines the scope of the holdout:
  
  'global': Affects all experiments across the entire project
  
  'local': Affects only specific experiments or flags
  Allowed
  globallocal
- start_timestring<date-time>
  Example
  2024-01-01T00:00:00.000Z
  Description
  Timestamp when the holdout became or will become active. When the holdout starts affecting traffic allocation.
  Format
  date-time
- statusstring<draft | running | concluded>
  Example
  running
  Description
  Current operational status of the holdout:
  
  'draft': Holdout is configured but not yet active
  
  'running': Holdout is currently active and affecting traffic
  
  'concluded': Holdout has finished and is no longer active
  Allowed
  draftrunningconcluded
- traffic_allocationinteger
  Example
  1000
  Description
  Percentage of traffic allocated to this holdout, expressed in basis points (1/100th of a percent). For example: 100 = 1%, 1000 = 10%, 5000 = 50%. The remaining traffic sees experiments normally.
  Minimum
  0
  Maximum
  10000
- updated_timestring<date-time>
  Example
  2024-01-01T10:00:00.000Z
  Description
  Timestamp when the holdout was last modified.
  Format
  date-time
- urlstring
  Example
  /holdouts/12345
  Description
  RESTful link to fetch this holdout's details. For project-scoped endpoints: /projects/{project_id}/holdouts/{holdout_id} For cross-project endpoints: /holdouts/{holdout_id}
- urnstring
  Example
  holdout.flag.optimizely.com::12345
  Description
  Universal Resource Name (URN) for the holdout, used for global identification.

Represents a holdout experiment that reserves a portion of traffic to measure the impact of feature flags and experiments.

Holdouts help establish control groups by excluding a percentage of users from seeing any experiments, allowing you to measure the overall impact of your experimentation program against a baseline.