Create a basic prioritization framework

Use a prioritization framework to evaluate your ideas and decide which experiments and campaigns to run first. A basic prioritization framework uses consistent criteria to order the experiments and campaigns to run from first to last. This helps you manage your backlog and experiment cycles and ensures that your most impactful tests run first.

This basic template for prioritizing ideas (.xlsx) can help you get started. A concept called minimum detectable effect (MDE) can also help you prioritize tests based on the expected ROI.

A full roadmap includes a scoring rubric and an execution timeline.

Materials to prepare

People and resources

Actions to perform 

Deliverables

What to watch out for

Evaluate ideas based on two factors: impact and effort. High impact or low effort depends on your company's business goals and your team's resources.

Impact – The metrics measuring the success of your optimization program and how Optimizely Experimentation events directly influence these metrics

Effort – What is easy or difficult to do, and what resources are dedicated to testing, including any shared or borrowed from other teams

Team members to consult

The criteria you use to prioritize your ideas depend on your program's goals and resources. For example, your team may have the technical resources to set up testing, but is low on mockup resources. Or, if you have executive buy-in but find it difficult to get time with your developers, you may find that advanced test ideas are approved but slow to be implemented.

These factors are important to consider when deciding on the criteria for effort and impact.

Assign effort and impact scores to every optimization idea to obtain a more granular view of each experiment's relative ROI. This is known as a blended model. Assign numerical values for effort and impact, then sum the impact and effort scores for each test and campaign to generate a single prioritization score.

You can use broad categories like high, medium, and low when evaluating impact and effort.

In the previous example, high-impact tests and campaigns have high numerical scores. However, high-effort ideas are given low numerical scores. Summing the two scores puts the high-impact, low-effort ideas at the top of the list. You should run these tests and campaigns first.

Creating a detailed rubric can further enforce consistency in prioritization. The effort scores should match your team's strengths. Adjust the impact scores based on business goals. 

If you have a mature optimization program with dedicated developer resources, you may be able to focus on tests based on impact without needing to weigh effort.

When you have prioritized your ideas and run a few, review how well your process works for your team. The following are two questions to consider:

When you prioritize your testing ideas, you can put them into a backlog or a full, prioritized roadmap. The first option is more flexible, but the second provides a stable cadence for organizing workflows.

A prioritized backlog is like a queue. When you finish one idea, go to the next. If you find an idea with great potential and want to focus your efforts there, you can.

With a full, prioritized roadmap, you commit to a timeline based on how long you think an experiment will run. Most mature programs prefer this approach, as it lets them coordinate stakeholders and schedule a complex workflow.

With a full roadmap, you plan your work in advance. You also build a regular cadence for incorporating insights and trends from completed tests and campaigns into a new round of testing. If you want to return to an idea, you can re-prioritize it and add a second iteration later.

Each method gives you insights from completed tests and campaigns to help you re-prioritize for the next test.

As you learn the cadence of your optimization program’s work cycle, evaluate whether you are trying to do too much or not enough.

Prioritizing 25 ideas but only executing five may mean you are over-prioritizing. Teams that over-prioritize consistently fail to implement lower roadmap ideas. Evaluate whether low-priority hypotheses are worth prioritizing again. If fewer ideas help you align your roadmap to your team's cadence, consider keeping the list short.

If you feel like you do not have enough high-impact ideas, you may be under-prioritizing. Teams that under-prioritize often run out of ideas before the cycle ends and return to ideation. Consider focusing your efforts on generating more ideas for your backlog. An idea submission form and a business intelligence report can help you increase the number of high-quality hypotheses.

Consider automating the idea submission process at your company. By automatically scoring the ideas submitted to your team, you can more easily evaluate and prioritize them.

Create an idea submission form that asks questions about the resources and skill sets required for a given idea. The responses to this form populate a spreadsheet with built-in formulas that add or subtract points based on the responses. You can then sort the ideas by scores.

A formalized process of collecting and prioritizing ideas focuses your team on running strong tests and campaigns and pursuing wins that generate excitement for experimentation. Publishing your automated submission form company-wide can also help to increase awareness of prioritization and optimization goals.

By equipping your team with a prioritized list of tests and campaigns, you build a sustainable and effective testing program.