Optimizely Personalization Campaign Results page
The Campaign Results page lets you track campaign performance, examine different experiences and audiences for a detailed view of visitor behavior, and take action.
- A session starts when a visitor lands on your site and ends after 30 minutes of inactivity. If there is new activity, a session is created.
- By default, Optimizely Personalization maintains a 5% holdback:
- 95% of visitors in a campaign are shown a personalized experience.
- 5% of visitors see the original.
From the Optimizations dashboard, click Results.
Optimizely Personalization dashboard
Optimizely Personalization + Optimizely Web Experimentation dashboard
From the Manage Campaign dashboard, click View Results.
Note
Results in Optimizely Personalization are in local time, according to the time zone set on your machine.
Campaign Summary
The Campaign Summary helps you track your campaign's overall impact. In the sidebar, you can view when the campaign was last published and how many days it has been running.
Overall Improvement
The Overall Improvement module provides a high-level view of your campaign's progress.
By default, the module shows you the improvement in the campaign's primary metric with a confidence interval. The improvement measures the relative difference between personalized sessions and the holdback (original) experience. It also tells you about the reach of campaign reach: the number of personalized sessions delivered to visitors and the impact of that metric per session.
Use the drop-down list to toggle between campaign metrics.
Filter by date range
Use the Date Range drop-down list to select start and end dates for your results page view. Click Apply. The results generated are in your computer's time zone.
Campaign Reach
The Campaign Reach module shows how many personalized experiences were delivered by session.
- Page reach – Shows the total number of sessions on pages targeted by your campaign.
- Audience reach – Shows the percentage of sessions delivered to one or more audiences in the campaign.
- Personalized sessions – Shows the number of sessions where a visitor saw a personalized experience.
- Holdback sessions – Shows the number of sessions where visitors saw the original experience because they are bucketed into the holdback by chance.
Experience Breakdown
The Experience Breakdown provides a visual summary of the impact of the campaign divided by audience.
The width (x-axis) of each bar shows the reach of a particular audience.
The height (y-axis) of each bar shows the change in conversion rate. For conversion events such as clicks or pageviews, the y-axis shows the conversion rate. Revenue is displayed as revenue per visitor.
The color of the bars shows whether the change in conversion rate is statistically significant:
- Green – A statistically significant lift.
- Red – A statistically significant loss.
- Gray – A change that has not reached statistical significance.
Note
See how Stats Engine calculates statistical significance.
To see more granular data about the primary metric by audience, click More Details.
The Impact column shows the difference in conversions in a personalized experience compared to the holdback.
Audience summary
The audience summary helps you track the improvement generated by the personalized experience for each audience by metric.
Select an audience and view a summary.
The summary always shows your primary metric first.
To view a graph of conversions in a specific metric for a particular audience, click View graph.
Change the baseline
Sometimes you may want to see how your variations compare to one variation in particular—which may not be the original. Use the Baseline drop-down list to select a different variation as the baseline.
Note
Changing the baseline should only be used for data exploration, not making decisions.
Segment results
Optimizely Personalization lets you segment your results by attributes like browser, device, campaign, and source to learn more about the behavior of certain groups of visitors.
Metrics definitions
- Sessions – A session starts when a visitor lands on your site, continues as they go between pages, and ends after 30 minutes of inactivity. The next visit creates a session. Formula: Absolute number
- Personalized sessions – The number of sessions a visitor was shown a personalized experience. This excludes visitors who do not qualify for any audience in the campaign or qualify for an audience but are randomly bucketed into the holdback (control group). Formula: Sum the treatment size over all audiences
- Reach – The percentage of total sessions that qualified for an audience (regardless of whether those visits were personalized). Formula: Number of sessions for an audience divided by total sessions
- Conversions – The number of sessions a visitor from any campaign audience converted on this event. Formula: Absolute number
- Conversion Rate – The percentage of sessions a visitor from any campaign audience converted on this event. Formula: The number of sessions where conversions took place for that audience or for total sessions.
- Holdback – The control of an A/B test. You can adjust the holdback for each campaign. See Holdback: Measure overall impact in Personalization for information. Formula: By default, Optimizely Personalization shows 95% of campaign visitors a personalized experience and holds back 5% who see the original.
- Confidence Interval – A range of values in which the conversion rate for a particular experience lies. As you collect more data, the confidence interval narrows. After your variation reaches statistical significance, the confidence interval is above or below 0. Formula: See Faster decisions you can trust for digital experimentation.
- Improvement – The relative improvement of this event's conversion rate for personalized sessions compared to the holdback. Formula: (Event conversion rate for personalized visits minus event conversion rate for control visits) divided by event conversion rate for control visits
- Improvements for an event for each audience – The % improvement in an event's conversion rate, comparing a personalized audience to the control. Formula: (Event conversion rate for audience's personalized visits minus event conversion rate for audience control visits) divided by event conversion rate for A=audience control visits
- Total Improvement in Primary Metric – The % improvement of the Primary Metric across all sessions in the campaign, regardless of whether they fell into an audience. Formula: (Conversion rate for sessions in the treatment minus conversion rate of sessions in control) divided by conversion rate of sessions in control
Add or remove metrics
Existing events and tags are tracked, so you can add them back. However, if you add more than five events, it may take longer to reach statistical significance.
- Click Manage Metrics.
- Click Add (+) to add metrics or drag metrics to remove them.
Set currency symbols
You can change the currency symbols displayed for revenue metrics with the Currency Picker under Project Settings > Advanced.
Changing the currency symbol does not convert revenue values based on an exchange rate; it only changes the symbol displayed. A revenue value displayed as $100.00 would become ¥100 after picking ¥, regardless of the current exchange rate between USD and JPY. Learn more about tracking revenue in Optimizely Personalization.
The currency symbol setting you select is remembered for the sessions on your current browser. Set the symbol again when logging in from a new device or browser.
How quickly do results update?
Results are typically available within 5 to 10 minutes of Optimizely Personalization receiving the data. See data freshnessdata freshness.
Reset or delete results
To remove visitors and conversions from your Results page, click Reset Results on the results page Campaign sidebar.
Note
Clicking Reset Results resets your results to the state of the last hourly refresh. Even after you click reset, you might see conversions and visitors from this timeframe who were part of your campaign.
Export results data
Use the Export CSV feature to download the results of your experiment in a comma-separated value (CSV) file. This file can view your results data in your favorite spreadsheet program. You can also share the raw results with others, store the data on your machine, or perform additional analysis.
Click the Export icon to download the CSV file of the results shown on the page (limited to the Date Range and Segment selected).
See Reference for columns in CSV export files for campaigns and experiments. You can also access your export data through an Amazon S3 bucket.