How Optimizely Experimentation counts conversions
- Optimizely Web Experimentation
- Optimizely Personalization
- Optimizely Performance Edge
- Optimizely Feature Experimentation
- Optimizely Full Stack (Legacy)
When sending a decision event to Optimizely Experimentation, how Optimizely assigns conversion events depends on the product.
Web Experimentation, Optimizely Performance Edge, Feature Experimentation, and Full Stack (Legacy)
- Uses user-level scoping for visitor count and event-level scoping for events.
- Conversion events are attributed to the first variation the user encounters, depending on the date range.
Optimizely Personalization
- Uses session-level scoping for both visitors and events.
- Conversion events are attributed to the last decision made in the session.
- This approach is ideal for behaviorally targeted campaigns with dynamic, evolving experiences.
When segmenting on the Optimizely Experiment Results page:
- The visitor count shows all visitors who had the attribute in their decision or had a conversion event with that attribute value.
- The conversions and events will only show events that have the attribute added to the event.
For information about how event counting works, see Attributes and segmentation.
Example
While you are experimenting on your site's product detail pages, a user visits one of those pages, enters the experiment, is bucketed into a certain variation, and converts on an event you track: Click Add to Cart. The user converts a few times while navigating your site. Unless otherwise noted, assume that the user is shown the same variation in Sessions 1 and 3.
- E = Conversion events (also known as events) are triggered when a visitor converts to a desirable action, such as a click, page view, or purchase.
- Conversion events in Optimizely Feature ExperimentationOptimizely Feature Experimentation.
- Conversion events in Optimizely Web Experimentation.
- Conversion events in Optimizely Personalization.
- D = Decision events (also known as impressions) special events fired when Optimizely Experimentation determines that a visitor is bucketed into a certain experiment or variation pair.
- Decision events in Optimizely Feature ExperimentationOptimizely Feature Experimentation.
- Decision events in Optimizely Web Experimentation.
- Session – A period of activity for a user. Optimizely Experimentation does not use sessions to assign data to decisions. Instead, sessions are used for illustrative purposes only. For example, Session 1 could be considered the user's activity on one day. Session 2 is the next day, and so on.
User-level scope
- Optimizely Web Experimentation
- Optimizely Performance Edge
- Optimizely Feature Experimentation
- Optimizely Full Stack (Legacy)
When a user is shown a variation, subsequent conversion events from that user are assigned to that variation for as long as the experiment is running.
When a user encounters multiple variations for the same experiment, subsequent conversions are assigned to the first variation encountered within the date range of the results query, meaning a user's conversions can be assigned to different variations, depending on the date range of your results query. For example, a user encountered Variation A at Time 1, then encountered Variation B at Time 2, and finally converted at Time 3. The conversion is assigned to Variation A if your results query spans from Time 1 to Time 3. If your results query spans from Time 2 to Time 3, the conversion is assigned to Variation B.
The following timeline shows how a single user's conversion events (E) are counted when results are scoped to the user.
Optimizely Web Experimentation counts 16 total conversion events (E) and assigns them to one user. This is a user-scoped event assignment. When the user is shown a variation, subsequent conversion events from that user are assigned to that variation.
- Users – 1
- Total conversion events scoped to the user – 16
- Unique converting users – 1
Session-level scope
- Optimizely Personalization
Session-level scoping in Optimizely Personalization assigns conversion events to the decision within that session while retaining the event scope for events when segmenting, making it suitable for behaviorally targeting with changing personalized experiences over time.
When the user is shown a variation, subsequent conversion events (E) that are triggered are assigned to that variation.
- experience_1 events – 3
- experience_1 unique converting events – 1
- experience_2 events – 2
- experience_2 unique converting events – 1
Decision-level scope
- Optimizely Web Experimentation
- Optimizely Personalization
Decision-level scope assigns each conversion event to the most recent unique decision made before that event occurred. Contextual bandits use this scope because visitors can be reassigned to a different variation while it is running.
When a visitor triggers a conversion event, Optimizely attributes that event to the variation from the "previous unique decision", even if the visitor had previously seen a different variation. This ensures that each event reflects the variation the visitor experienced at the time of the action.
For example, User123 is exposed to Variation_2 and triggers the following four different conversion events:
- Add_to_bag
- Product_viewed
- Bag_viewed
- Removed_item
Then, User123 is exposed to Variation_1 and triggers the following two conversion events:
- Checkout_1
- Revenue
The Contextual Bandit Results page in the previous example shows the following:
Variation_2
- Decisions – 1
- Events – 4 (Add_to_bag, Product_viewed, Bag_viewed, and Removed_item)
Variation_1
- Decisions – 1
- Events – 2 (Checkout_1 and Revenue)
If User123 is later reassigned to Variation_1, Optimizely does not increment the decision count because User123 already received a decision for that variation. However, additional conversion events are still counted and attributed to Variation_1.
Key benefits of decision-level scope
- Accurate attribution – Events are precisely tied to the context (variation) in which they occurred, even when user context changes mid-campaign.
- No inflated counts – Users who see the same variation multiple times do not increase decision counts, helping avoid misleading aggregation.
- Consistent reporting – This approach mirrors event-scoped segmentation, where event attribution depends on when the event occurred relative to the decision.