Segment your results
- Optimizely Web Experimentation
- Optimizely Personalization
By default, Optimizely Web Experimentation and Optimizely Personalization show results for all visitors who enter your experiment. However, not all visitors behave like your average visitors. Segmenting lets you filter your unarchived experiment results to see if certain groups behave differently from your visitors overall.
You can filter results in the Optimizely Experiment Results pageOptimizely Experiment Results page and Optimizely Personalization Campaign Results page with the following default segments that Optimizely captures automatically:
- Browser type
- Campaign
- Device type
- Referrer
- Source type
You can also add custom attributescustom attributes to your experiments and campaigns and use these attributes for custom segmentation to get a more granular view of your business's most important visitors.
For example, you run an experiment that tests a pop-up promotional offer. This generates a positive lift overall, but segmenting visitors on mobile devices is a statistically significant loss. Maybe the pop-up is disruptive on a mobile device or difficult to close. When you implement the change or run a similar test in the future, you might consider excluding mobile visitors based on what you have learned.
Segmenting results helps you see beyond the average visitor's behavior and improve your experimentation program.
Note
Optimizely Web Experimentation does not offer segmentation by audience.
Default segments
Optimizely Web Experimentation and Optimizely Personalization capture several default attributes that you can use to segment your Results page.
Important
You must have at least one visitor assigned to an attribute for it to display on your Results page.
- Browser type
- Firefox
- Google Chrome
- Internet Explorer
- Opera
- Safari
- Unknown
- Campaign – Visitors who enter the experiment with the parameter utm_campaign capture the parameter value. Use this value to examine campaign keywords like holiday_sale.
Note
There is a 20-character limit for values. Longer values are truncated.
- Device type
- Desktop/laptop – Visitors rising from a desktop or laptop.
- All phones – Visitors using any mobile phone. It is a combination of the iPhone and Other phone segments.
- iPhone – Apple iPhone visitors
- Other phones – Visitors using a mobile phone that is not an iPhone.
- All tablets – Visitors using any tablet. It is a combination of the iPad and Other tablets segments.
- iPad * – Visitors using iPads.
- Other tablets – Visitors using tablets that are not iPads.
- Other devices – Visitors using devices not identified as desktops, laptops, phones, or tablets.
- Desktop (deprecated) – Visitors using a desktop, laptop, or other device. Use the desktop/laptop segment to focus on visitors using traditional desktop browsers.
- Referrer – Initial referrer or the originating URL (the URL of a page outside the domain of the snippet that the visitor navigated from before landing on a page where the snippet is implemented). For example, if a visitor navigates to your domain by clicking on a link on https://www.facebook.com/search/top, events triggered during the session have a referrer value of https://www.facebook.com/search/top.
- Source type
- Campaign – Visitors that arrive on a URL containing a utm_campaign, utm_source, gclid, or otm_source query parameter. If the URL contains one of these parameters, the visitor counts as Campaign traffic even if they arrived through Search.
- Search – Visitors that visit through a referral URL containing Bing, Google, Yahoo, or Baidu but not a specific campaign parameter (otherwise, they would be bucketed as Campaign).
- Referral – Visitors from another URL that do not count as Campaign or Search.
- Direct – Visitors who do not have any external referrer in their URL.
Note
*Apple changed the user agent for iPad in iOS 13 such that it displays as Desktop/laptop traffic in Optimizely and other analytics tools. This may increase Desktop/laptop traffic and decrease iPad traffic when segmenting your results.
The Source type* *value is based on the document.referrer value in the browser, except Campaign. The visitor Source type value may change as they navigate your site, such as a checkout flow with a step where the visitor goes to a third-party site or different subdomain and then returns. Visitors would have an original Source type value (Campaign, Referral, or Direct) until they leave the site. When the visitor returns, they have a referral value because Optimizely detects that they went to another site.
You may see the same visitor's conversions in segments for different Source type values on your Results page. For example, conversions that the visitor made before leaving the site display in Campaign or Direct. Conversions triggered after the visitor returns show in Referral without affecting the earlier conversions.
Custom segmentation
You can create custom attributescreate custom attributes and use them for custom segmentation (with more granularity) on the Results page. Custom attributes describe key characteristics of your different visitors. Use them to segment your Results page in the exact same way as default segments.
For example, you might create a custom attribute that targets by plan type, which includes the following values: basic, plus, and premium. You can use these to segment your results and identify when visitors with a basic plan behave differently from visitors with a premium plan.
Segment the Results page
You can segment your Optimizely Experiment Results pageOptimizely Experiment Results page and Campaign Results page. Segmenting results help you get more out of your data by generating valuable insights about your visitors.
Note
Segments and filters should only be used for data exploration, not making decisions.
In the Segment drop-down list, you see default segments and custom segments for any attributes assigned to at least one visitor in the experiment.
- Go to your Results page.
- Click Segment and select one or more attribute values from the drop-down list.
- Choose Match All Values or Match Any Value if selecting multiple.
- Click Apply.
Match All Values and Match Any Value
When you select more than one attribute value from the segmentation menu, the Results page can display visitors who match all values or visitors who match one or more values. The radio buttons control this behavior, labeled Match All Values and Match Any Value.
For example, you implement two custom attributes to track a visitor's location and plan type. If you want to see how visitors in the United States (US) with a Gold plan responded to your variations, your segmentation configuration looks like the following:
Match All Values
- Location – US
- Plan Type – Gold
If you select Match Any Value instead, the Results page displays data for visitors who are in the US combined with data for visitors who have a Gold plan.
How visitors are counted in segments
For information on how Optimizely counts conversions when segmenting, see the following:
- How Optimizely counts conversionsHow Optimizely counts conversions
- How attributes and segmentation workHow attributes and segmentation work
Geo results segmentation
If your Optimizely plan includes geo-location for audiences, you can use Optimizely's geo-location script to obtain location information, fetch geo data using Optimizely's (Akamai's) geolocation service, and pass it as a custom attribute in Web Experimentation for segmentation, such as segmenting results by visitors in the US.
Prerequisites
To use geolocation features, ensure you have a global or site-wide experiment running with a location-based audience, such as an A/A testA/A test. This triggers the geo.js script, which fetches the visitor's location.
To load the geolocation snippet (geo.js)
- Ensure at least one experiment uses a geo-location audience. Without this, the visitor's location is not fetched.
- Run a dummy experiment with a geo-audience, even if you do not plan to use this information for any audiences.
- Ensure the dummy experiment runs on pages where you need geolocation data. This lets you track events for the experiment and segment your audience effectively.
Steps
The following steps describe how to retrieve the country from the location object and pass it to a custom location attribute.
- Create an experiment with an audience that targets a location (any location). This ensures that the geo.js script is loaded on the page, which in turn adds a location object to the Optimizely visitor object.
- Create custom attributes to leverage the available geo parameters, such as city, continent, country, Designated Market Area (DMA), and region.
- Add the relevant code snippet to your Project JavaScript to subscribe to the onActivated notification listener. See the the developer documentation on activated activated for information.
window.optimizely = window.optimizely || [];
function onActivated() {
let visitor = window.optimizely.get('visitor');
let country = "";
if (visitor.location.country) {
window.optimizely.push({
"type": "user",
"attributes": {
"location": visitor.location.country,
}
});
}
}
window.optimizely.push({
type: "addListener",
filter: {
type: "lifecycle",
name: "activated"
},
handler: onActivated
});- Define your attributes within the callback function.
let visitor = window.optimizely.get('visitor');
let country = "";
if (visitor.location.country) {
window.optimizely.push({
"type": "user",
"attributes": {
"location": visitor.location.country,
}
});
}Now, you should be able to view the segmented results in the Results page.
Note
If you use the Chrome DevToolsChrome DevTools to change your location using Sensors, the browser's global navigator object shows this new location. However, the geo.js script does not recognize this change. Instead, it continues to use the original location data, ignoring the simulated changes made in the developer tools. See Sensors: Emulate device sensors in the Google documentation for information.