Warehouse-Native Experimentation Analytics
Get started with Warehouse-Native Experimentation Analytics
Optimizely Warehouse-Native Experimentation Analytics enhances your experimentation results by evaluating experiments against data that lives in your data warehouse. This ensures your data warehouse remains the single source of truth, keeping your data secure and centralized.
Experiment analysis
In Optimizely, experiments are structured tests designed to compare different variations of a webpage, feature, or experience to determine which performs best based on specific goals. These experiments help businesses optimize user experiences, increase conversions, and make data-driven decisions.
Experiment Diagnostics tab
Use the Diagnostics tab to confirm that an experiment reaches users and records conversions before you trust its results. The tab shows the raw decision and conversion events behind the experiment and checks the integrity of its data.
Experiment readouts
Use an experiment readout to share an experiment's results with stakeholders on one page. A readout is a dashboard with a verdict, the scorecard, and a chart for each metric.
Create a decision dataset in Analytics
Decision events (or impressions) are special events triggered when Optimizely Experimentation "decides" that a visitor is bucketed into a certain experiment and variation pair. See Data specification https //docs.
Conversion metric
Use a conversion block in Warehouse-Native Experimentation Analytics to segment a dataset based on the observed behavior and other properties. The behavior of a dataset's record is determined by the events that are linked with the entity.
Numeric aggregation metric
Numeric aggregation in Warehouse-Native Experimentation Analytics lets you create simple aggregations over existing columns in your data. A numeric aggregate block calculates a summary of a property or block.
Ratio metric
In Warehouse-Native Experimentation Analytics, a ratio metric divides the values of two events to produce a single derived value. You can use it to measure derived rates such as content completion rate, revenue per session, or items purchased per visit.
Create an Experiment Scorecard in Optimizely Analytics
Complete the following to create an experiment scorecard in Warehouse-Native Experimentation Analytics Add your Optimizely account ID in the Warehouse-Native Analytics app settings. To add your account ID, contact Optimizely Support https //support.
CUPED (Controlled-experiment Using Pre-Experiment Data)
CUPED (Controlled-experiment Using Pre-Experiment Data) is a statistical method that reduces variance in A/B tests, enhancing their sensitivity and making it easier to detect differences between groups. By lowering variance, CUPED lets experiments achieve statistical significance with less data if there is a true treatment effect.
Statistical significance notifications
Statistical significance notifications tell you when a metric on your experiment reaches statistical significance. Optimizely watches every metric on the results page and notifies the users you choose by email and Slack.
Configure guardrail alerts in Optimizely Analytics
Guardrail alerts let you configure metric alerts that protect business-important metrics during an experiment. Each alert watches a metric you choose and notifies you when a variation moves that metric past a threshold you define, relative to the baseline.