Experiment Best Practices skill
The Experiment Best Practices skill is an org skill in the Optimizely Opal Skills Directory that applies your organization's testing methodology and quality standards so every experiment is designed, run, and analyzed the way best-performing teams do it.
It applies to everyone in your organization. An administrator installs and configures it once.
Skill details
- Shortcut – /experiment-best-practices
- Type – Org skill
- Setup – Required
What the skill does
- Gives everyone one consistent method for A/B, multivariate, personalization, and feature-flag tests.
- Scores hypotheses on a 10-point rubric, checks for duplicate and colliding tests, and calculates test duration in whole weeks.
- Provides a pre-launch quality gate, monitoring tiers, results readouts, backlog scoring, and program-level learning.
- Never fabricates baselines or results, never approves or launches a test, and never declares a winner without validity and guardrail checks.
Before you begin
- You must have the Manage Instructions permission to install this skill. If you do not, click Request in the Skills Directory to ask an administrator to install it.
- Opal works best with a connected experimentation or analytics tool, or an uploaded test log.
- Have your answers and files ready. Opal asks for them during setup.
Information Opal asks for
Opal asks for the following during setup. You can skip optional items.
- What You Test, Where, and How Much Traffic (Required) – The surfaces you test, the conversion that matters, and your rough traffic.
- Your Testing Standards (Optional) – One rule per line. Defaults are 95% significance (never below 90%), a 7-day minimum in whole weeks, and a 30-day maximum.
- A Past Test Brief or Readout (Optional) – Sets the format for plans and readouts.
- Your Test Log (Optional) – Powers the duplicate check and backlog scoring.
Install the skill
- Go to Directory > Skills.
- Find Experiment Best Practices and click Install.
- In the Setup required dialog, click Set up in chat.
- Answer Opal's questions and upload any files it asks for.
- Confirm the details when Opal asks.
For details about browsing, installing, updating, and uninstalling skills, see Skills Directory overview.
Use the skill
After setup, enter /experiment-best-practices in Opal Chat to run the skill on demand. Opal also applies it automatically when your request matches its When to use conditions. Try prompts like the following:
- How long should this A/B test run?
- Review this test plan before launch.
- Read out the results of this test. Is it a winner?
Things to know
- The duration formula suits binary conversion metrics. For revenue or other continuous metrics, use your tool's calculator or provide a standard deviation.
- If your program uses fixed-horizon or Bayesian methods, name the method in Your Testing Standards.
Update the skill
When Optimizely publishes a new version, Update displays on the skill's card in the Skills Directory. Opal keeps the information you provided during setup and asks only for anything the new version adds. See Update a skill.
Related skills
Note
If you use Opti ID, administrators can turn off generative AI in the Opti ID Admin Center. See Turn generative AI off across Optimizely applicationsTurn generative AI off across Optimizely applications.