Configure self-service A/B testing
You can conduct A/B testing for your product recommendation widgets in Optimizely Product Recommendations using the self-service A/B testing feature. This feature lets you independently create, configure, and launch experiments without relying on developer support. You can define variants (for example, different recommendation algorithms), assign traffic splits, and measure performance based on key metrics such as sessions, number of unique visitors, conversion rate, orders, or revenue.
By using self-service testing, you gain faster experimentation cycles, better control over your test configuration, and data-driven insights into which recommendation strategies perform best for your audience.
Create an A/B test
To create an A/B test, complete the following:
- Go to Configuration > A/B Tests.
- Click Create New Test.
- Enter a Name for your A/B test and add a hypothesis or a test description in the Description field.
- Click Create Test. The Variations page displays.
Add and edit variations
By default, you have Variation A and Variation B as two test groups of widgets you can use to conduct your A/B test. You can add up to five variations. These are the widgets you previously added to pages to enable and start generating product recommendations. See Activate a widget.
To edit a variation, complete the following:
- Click the name of your variation (like Variation A) or click Edit. The edit page displays.
- Search for and select widgets for each page where you want product recommendations. You can also create a widget by clicking + Create new widget or delete a widget by clicking More (...) > Delete.
- Click a widget to configure it.
- Click Save to save your changes.
- Go back to the Variations page to control the Total Traffic for each variation. Click More (...) to Rename, Duplicate, or Delete any variation.
- Click Save to start your A/B test.
Start and schedule your test
After adding your variations, you can immediately start your A/B test or schedule a date to start your test. To start your A/B test, click Start Test.
Note
You can only conduct one live test at a time. If you want to start a new one, go to your existing live test and click End Test.
To schedule a test, complete the following:
- Click Schedule.
- Select your Start Date, Start Time, End Date, and End Time. Select your Time zone from the drop-down list.
- Click Save. Your test starts and ends at the scheduled times.
View test results
You can view your test results when the test completes. To view your results, complete the following:
- Go to Reports > A/B tests or select a test from your list of completed tests (with the status Done).
- Click Results.
- Select your Control group (the variation whose data you want to see), Currency (the currency you want to examine), and Start Date and End Date.
- Click Numbers to view the comparison between your variations for key metrics, such as Sessions, Unique visitors, Revenue, Orders, and so on.
- Click Daily charts to view the data in a chart format. You can view the data for Revenue, Visitor-to-order conversion, Average order value, Average selling price, Unit per order, and Sessions.
- Click Uplift charts to view the individual and cumulative uplift for each variation.
- Click Widget Breakdown to view total revenue by variation. Expand a widget under a variation, for example, Variation D > ab_AWidget91, to view key metrics including Product refCode, Brand, Impressions, Clicks, Items Bought, and Revenue.
