Experiment Best Practices skill
The Experiment Best Practices skill is an organization skill from the Skills Directory in Mark (Optimizely Agent Platform). It applies your organization's testing methodology and quality standards. Every experiment is then designed, run, and analyzed the way best-performing teams do it.
The skill applies to everyone in your organization when you choose Organization during configuration. Creating an organization skill requires the Manage Skills permission.
Skill details
The following details apply to this skill:
- Shortcut – /experiment-best-practices
- Type – Organization skill
What the skill does
The skill does the following:
- Gives everyone one consistent method for A/B, multivariate, personalization, and feature-flag tests.
- Scores hypotheses on a 10-point rubric, checks for duplicate and colliding tests, and calculates test duration in whole weeks.
- Provides a pre-launch quality gate, monitoring tiers, results readouts, backlog scoring, and program-level learning.
- Never fabricates baselines or results. Never approves or launches a test. Never declares a winner without validity and guardrail checks. Guardrail checks confirm that a test does not harm your key metrics.
Before you begin
Confirm the following before you install the skill:
- You must have the Manage Skills permission to install this skill. If you do not, click Request in the Skills Directory to ask an administrator to install it.
- Mark works best with a connected experimentation or analytics tool, or an uploaded test log.
- Have your answers and files ready. Mark asks for them during configuration.
Information Mark asks for
Mark asks for the following during configuration. Skip optional items if you do not need them.
- What You Test, Where, and How Much Traffic – The surfaces you test, the conversion that matters, and your rough traffic.
- (Optional) Your Testing Standards – One rule per line. Defaults are a seven-day minimum in whole weeks and a 30-day maximum. The default significance level is 95% (never below 90%). Significance is the confidence needed to call a result real.
- (Optional) A Past Test Brief or Readout – Sets the format for plans and readouts.
- (Optional) Your Test Log – Powers the duplicate check and backlog scoring.
Install the skill
Go to Directory > Skills, find Experiment Best Practices, and click Install. In the Setup required dialog, click Set up in chat. Mark asks who the skill is for, Myself (Personal) or Organization. Mark then asks whether to answer the remaining questions in chat or upload a document. For the full procedure, see Install a skill that needs configuration.
Use the skill
After configuration, enter /experiment-best-practices in Mark chat to run the skill on demand. Mark also applies it automatically when your request matches its When to use conditions. Try prompts like the following:
- How long should this A/B test run?
- Review this test plan before launch.
- Read out the results of this test. Is it a winner?
Things to know
Keep the following in mind:
- The duration formula suits metrics with a yes-or-no outcome per visitor, such as signups. For revenue or other continuous metrics, use your tool's calculator or provide a standard deviation.
- If your program uses fixed-horizon or Bayesian methods, name the method in Your Testing Standards.
Update the skill
When Optimizely publishes an updated version, Update displays on the skill's card in the Skills Directory. Mark keeps the information you provided during configuration and asks only for anything the updated version adds. To correct answers that are no longer accurate, edit the installed skill. See Update a skill and Manage skills.
Note
If you use Opti ID, administrators can turn off generative AI in the Opti ID Admin Center. See Turn generative AI off across Optimizely applications.