Use a business intelligence report to ask the right questions
- Optimizely Web Experimentation
- Optimizely Personalization
- Optimizely Performance Edge
- Optimizely Feature Experimentation
- Optimizely Full Stack (Legacy)
Note
This article is part of The Optimization Methodology series.
A business intelligence report helps you build structure and consistency in experience optimization. It gathers key insights about your site and industry into a single document. When you brainstorm hypotheses, the report helps foster creativity and ground your team's idea generation in data.
Ideation, generating meaningful hypotheses for experimentation, is one of the most important but hardest steps in experience optimization. Without a structured framework supporting the creative stage of optimization, generating impactful ideas consistently can be difficult. Use insights from your report to generate ideas for experimentation.
Materials to prepare
People and Resources
- Program Manager
- Design resources
- Marketer
- Analysts
Setup
- Collect data for the intelligence report
- Analyze your data
What you end up with
- A business intelligence report
Notes
- Remember to perform this process year-round.
- Do not neglect to use this report for decision-making about the site instead of just ideation.
What to include
Direct data, indirect data, and goal trees provide different perspectives on your site experience.
- Direct data (your site) – Web analytics, results from prior experiments, voice of the customer, heat maps, user testing, user personas, customer decision models, lifetime value (LTV), and customer acquisition cost (CAC).
- Indirect data (industry insights) – Competitors' sites, best-in-class experiences, blogs, webinars, user groups, consultations, books, and academic research.
- Goal maps – Goals that align with your company so your experiments improve metrics that matter.
These information types provide insights beyond what you might see in a single source.
For example, if your web analytics report (direct data) reveals that visitors drop out of the funnel on a certain page, you may decide to test a clearer CTA to push visitors further down the funnel. While reviewing competitors' sites (indirect data), you notice another site personalizing their CTA. You decide that this tactic has the potential to be highly impactful and test whether personalizing the CTA works with your visitors.
In this example, the direct source (web analytics) tells you where to optimize. Indirect sources complement this data by suggesting how you might approach this opportunity. When used together, different sources explain questions surfaced by another data source.
Build the report
The comprehensiveness and depth of the intelligence report depend on your program’s resources and maturity level. Any program, regardless of size, can use the report to strengthen ideation.
- When will you produce your report? – Create a timeline with milestones. Plan to finalize the report before your next major planning session.
- What data sources do you use? – Identify the sources that your organization uses. Use the list above to identify potential resources that you can incorporate with team constraints.
- Who are your stakeholders for this report? – Assign report responsibilities to internal and external stakeholders. For example, ask the marketing team to review value propositions on competing sites or the web analyst to provide funnel segmentation analysis.
- What are the key insights did you find? – Identify a few from each data source to include in the report. Summarize these insights and gather them into a single deliverable.
- How will you use your report? – Introduce the report and talk about it in your planning meetings. Use it to generate ideas for experimentation and keep revising and reviewing it at a regular cadence.
Best practices
Review at a regular cadence
Prepare your business intelligence report for review at major team planning and decision-making sessions. Depending on how often your team meets for ideation, do this bi-weekly, quarterly, or monthly.
Focus on stories and insights
Fill it with meaningful insights and data about your visitors. An intelligence review should feed your team's creative process; use it to bridge the gap between export data from experimentation events and a well-designed hypothesis.
Use takeaways, summaries, and quick insights to tell stories about your customers' desires, optimization opportunities, and competitive advantages. Leave out metrics for performance reports about KPIs and revenue; this information can distract in ideation.