Calculate the statistical likelihoods of variations using Stats Engine

When you run experiments, Optimizely determines the statistical likelihood of each variation actually leading to more conversions on your metrics. This matters because when you look at your results, you are probably less interested in seeing how a variation compared to the baseline and more interested in predicting whether a variation will be better than the baseline when implemented in the future.

Optimizely’s Stats Engine powers statistical significance calculations. It uses an optimized statistical framework to run experiments with high statistical rigor while making it easy for anyone to interpret results. Specifically, Stats Engine lets you make business decisions on results as experiments are running, regardless of preset sample sizes and the number of goals and variations in an experiment.

As with all statistical calculations, it is impossible to predict a variation's lift with certainty. This is why the Optimizely Experiment Results page displays Optimizely's level of confidence. This way, you can make sophisticated business decisions from your results without needing an expert level of statistical knowledge.

To learn about the nuts and bolts of how Stats Engine works and how to interpret the results it generates, review these articles:

You may also find it useful to read about the statistical concepts and techniques Stats Engine uses:

If you would like even more background on how Optimizely's Stats Engine works, here are a few resources: