A/B tests in Feature Experimentation overview

Optimizely Feature Experimentation gives you the power of traditional A/B tests with the additional advantage of feature flagsfeature flags. Because your experiments are created on top of feature flags using flag rulesflag rules, they fit seamlessly into your software development lifecycle.

Experiments let you go far beyond simple "flag on" or "flag off" comparisons. Experiments help you answer and tackle difficult questions like, "Which feature or component should we invest in building?". In contrast, flag deliveries (targeted deliveries)flag deliveries (targeted deliveries) are for when you are confident in that answer.

For example, you are building a product sorting feature. With experiments, you could build flag variablesflag variables for different versions of the sorting algorithm, a different label in the interface, and different numbers of catalog items. You can test each variable in variations concurrently without deploying new code. Run an A/B test to learn how these variations perform with your user base, pick the winning variation, and roll it out with a feature flag.

You can shorten your market feedback time by comparing two variations of a flag in a controlled experiment in a given time frame, then view analytics on which variation was most successful right in the Optimizely app. You can interpret your metrics with the help of Optimizely's sophisticated Stats EngineStats Engine, built for the modern world of business decision-making.