Attributes and segmentation

How Optimizely Experimentation sets attributes of decision events differs depending on the scenario and what Experimentation product you use. This article considers examples of assigning events in various circumstances.

This example demonstrates how attribution and segmentation work in Optimizely Experimentation.

While you are experimenting on your site's product detail pages, a user visits one of those pages, enters the experiment, is bucketed into a certain variation, and converts on an event you track: Click Add to Cart. The user converts a few times while navigating your site. Unless otherwise noted, assume that the user is shown the same variation in Sessions 1 and 3.

Optimizely Experimentation lets you automatically target experiments and personalized experiences to groups of users who share attributes you define (called an audience or are defined by the Web Experimentation snippet). This section explains how Optimizely Experimentation attributes a single user's events over time as they go through the site.

When a user first visits the site, Optimizely Experimentation buckets them as a "logged-in visitor" and shows them a variation during Session 1. When the user returns on day three, they trigger another decision for that experiment in Session 3, and they no longer see the variation because they are now logged out and do not qualify for the experiment.

User-scoped Experimentation products assign events from this user from the moment they are bucketed (Session 1). After no longer qualifying for the experiment, events are still assigned to the user, but they will not see the variation or experiment.

Purchase event example for Experimentation

For example, a visitor to your site is bucketed into a variation during Session 1. Suppose this visitor is removed from the experiment in Session 3 by logging out. Afterward, they make a purchase. The visitor will no longer see the experiment, but events will continue to be assigned to the variation they were exposed to, as this change influenced them during their journey.

Purchase event example for Optimizely Personalization

Session-scoped products (Optimizely Personalization) assign events to the latest decision event.

For example, a visitor to your site qualifies for a personalization campaign and is bucketed into an experience during Session 1. Suppose this visitor makes a purchase in Session 3 after logging out and qualifying for a different experience. The events from the first experience are assigned to the first experience, and the events from the second experience are assigned to the second experience.

In the preceding diagram, the red events are assigned to experience_1. The blue events are assigned to experience_2.

Decision-level scoped products (contextual banditscontextual bandits) events are assigned to the decision in which they happened. Contextual bandits use decision-level scope because visitors can be bucketed into different variations within the same session. 

Consider the following example:

Using decision-level scope, the Contextual Bandit Results pageContextual Bandit Results page displays the following:

When you look at your results in Experimentation, you can use segmentation in Web Experimentationsegmentation in Web Experimentation or Feature ExperimentationFeature Experimentation to see how a subset of users respond to the variation on the results page.

Result segments for the events are event-scoped, meaning segmentation and attribution are based on the event's value. Event scoping lets you filter and analyze data based on the context and details provided by individual events. The visitor count is based on the decision attributes, and the event metrics are based on the events' attribute values.

When segmenting on the Experimentation results page:

Event-based segmentation for registration example

A visitor browses an ecommerce site and adds items to their cart as a guest (not logged in). They create an account at the checkout (logged in). 

Optimizely segments the events accordingly:

This clear separation lets you conduct a more accurate user journey analysis, ensuring each event is associated with its accurate logged-in state.

Event-based segmentation for membership tiers example

For example, imagine you are an ecommerce platform user and want to assess users' shopping behavior across different membership levels: Basic, Premium, and Elite. 

A user with a Basic membership logs in and browses your curated selection of products. During their browsing session, they encounter a promotion, enticing them to upgrade to the Premium membership for an exclusive discount. They decide to upgrade. 

This precision in segmentation ensures that each user action is correctly associated with its respective membership level, allowing for granular analysis of user behavior across different tiers. This approach provides insights to guide marketing and UX strategies for each membership category.

Multiple attributes in events example

A user sends five events; three have the attribute not logged in, and four have logged in.

When you segment your results for:

Event-based segmentation lets you view detailed insight into events, ensuring you see each event in its correct context without attribute overlap or confusion.

The same event-scoped segmentation logic applies to Optimizely Personalization. The visitor count is based on decision attributes, and event metrics are based on the events' attribute values. Attributes available for filtering include any attribute found on the decision event, the conversion event, or both.

When segmenting on the Optimizely Personalization Campaign Results pageOptimizely Personalization Campaign Results page:

Unlike experiments, which assign events to the visitor's first decision (user-scope), Optimizely Personalization assigns events to the most recent decision event within the visitor's session (session-scope), because a visitor may qualify for multiple experiences during the lifetime of the campaign.

Session-scoped segmentation example for Optimizely Personalization

A visitor qualifies for two experiences during the same session. They qualify for experience_1 with the attribute is_logged_in = True, and later qualify for experience_2 with the attribute is_logged_in = False.

When you filter by is_logged_in = False:

In Optimizely Experimentation, you can also filter your results by date range. For example, you might want to see the conversions that occurred between January 1 and January 20 on your results page. This section explains how Optimizely Experimentation attributes conversion events when filtering your results by date range.

The counting rule for date ranges applies when a decision is made during the filter's date range, and then events in that session are counted when that date range filter results.

Date range example for Optimizely Experimentation products

Optimizely looks at the first decision event of that user within the date range. Events after but within the date range are assigned. Events outside the date range and events assigned to a decision outside the date range are not counted.

Date range example for Optimizely Personalization

When a user is bucketed into the personalization campaign, and the session starts in the date range (Session 3), subsequent conversion events that occur during that user session are assigned to the last decision event. 

Unlike experiments where events are assigned to the 'first decision,' personalization campaigns consider the 'last decision' to determine to which experience the events should be assigned. This distinction is crucial because users can change states within a campaign, adding a layer of complexity compared to standard experiments.

In Optimizely Experimentation, you can reset your results page. This section explains how a user's events are assigned to an experiment or campaign when you reset the page.

When you reset your results, events that began before the reset point are discarded. Events after the results reset and after a decision event are counted.

Resetting results example 

You see event-tracking calls for visitors to your page when monitoring your network traffic. When a visitor triggers an event in Optimizely Web Experimentation, the event fires a tracking call picked up in network traffic.