Monitor monthly active users (MAUs)
- Optimizely Feature Experimentation
- Optimizely Full Stack (Legacy)
- Optimizely Web Experimentation
- Optimizely Personalization
- Optimizely Performance Edge
Your monthly active users (MAUs) are the number of unique users used in a decision or tracking event. It is a measure of the overall traffic where you are using Optimizely Experimentation's snippet, APIs, or SDKs to do something, specifically one of the following:
- Experiment evaluation.
- Personalization campaign evaluation.
- Event tracking.
- Any page where the Optimizely Web Experimentation snippetOptimizely Web Experimentation snippet is loaded, unless restricted by holdEventsholdEvents or optOut features.
Unlike impressions, it is not a measure of what percent of that traffic you are experimenting on. Every user who gets evaluated is counted. This lets you run large-scale experiments at 100% traffic to reach statistical significance.
This article describes how MAUs work at Optimizely Experimentation from a technical perspective.
MAUs in Optimizely Feature Experimentation
In Optimizely Feature ExperimentationOptimizely Feature Experimentation, an MAU is counted once per month for each unique user ID that triggers at least one decisiondecision or trackingtracking event.
Note
MAUs are not counted for decision events for disabled flags and targeted delivery rules, but this may change in the future. This is because a decision event (impression) is not sent when the Decide method is called.
MAUs in Optimizely Full Stack (legacy)
In Optimizely Full StackOptimizely Full Stack, an MAU is counted each time a decision or tracking event is sent for a unique user ID:
- When the optimizelyClientInstance.activate() method is used, and the visitor is bucketed into a variation of an A/B test (experiment evaluation).
- When the optimizelyClientInstance.isFeatureEnabled() method is used, and the visitor is bucketed into a feature test (feature flag or rollout evaluation).
- When the optimizelyClientInstance.track() method is used (tracking event).
Users are counted even if they receive a disabled flag due to activate() or isFeatureEnabled() because a decision event was made.
MAUs in Optimizely Web Experimentation
In Optimizely Web Experimentation, each unique visitor who encounters a page where the snippet is loaded is counted as an MAU, even if the snippet is empty and does not contain page activationspage activations, eventsevents, or experiment evaluationsexperiment evaluations.
If you do not want Optimizely Web Experimentation to run on a specific page, remove or disable the snippet using the relevant JavaScript call in Project JavaScript.
De-duplication
As long as it is the same unique user ID, Optimizely Web Experimentation de-duplicates. For example, if a user goes to a page with one project's snippet and then a page with another project's snippet, only one MAU is counted as long as it is the same unique user ID.
Note
If you use Optimizely Web Experimentation and Optimizely Feature Experimentation, you can override anonymous Optimizely Web Experimentation user IDs with known Optimizely Feature Experimentation user IDs to avoid overcounting. For information on including Web IDs, see the documentation on Bring your own visitor ID (BYOID)Bring your own visitor ID (BYOID).
Decision event
When an Optimizely Feature Experimentation experiment or a page within an Optimizely Web Experimentation experiment is activated, a decision request is sent. Decision requests look like this:
In the request payload, the decision attribute indicates the experiment that it applies to.
Example
The following scenario has three multipliers:
- Experiments
- Pages (as defined in Optimizely Experimentation)
- Pageviews
The Attic and Button company is experimenting on www.atticandbutton.us. Consider a visitor who starts by visiting the Attic and Button homepage, where there are three experiments running. One of these experiments has two Optimizely Experimentation pagesOptimizely Experimentation pages that both target the homepage:
Experiment 1
- Homepage – https://www.atticandbutton.us/ (simple matchsimple match)
Experiment 2
- Homepage – https://www.atticandbutton.us/ (simple match)
Experiment 3
- Homepage – https://www.atticandbutton.us/ (simple match)
- Global page (a page defined in Optimizely Experimentation that targets all URLs of a website) – https://www.atticandbutton.us/ (substring matchsubstring match)
Three bucketing decisions are made on whether to include that user in the experiment or not, but because the user ID is unique, this only counts towards one MAU. This is in contrast to impressions, which would have counted four impressions for this one user.
If the visitor refreshes the page, they still only count as one MAU. This is in contrast to impressions, which would have generated another four impressions, making the total eight impressions for this one particular user.
Now, suppose that you are running a search algorithm experiment with Optimizely Feature Experimentation on the homepage, too. When a visitor types a search term, the results are refreshed without reloading the page. The Optimizely Feature Experimentation SDK makes a decision for a variation every time a new search is done. This means that if a visitor searches for "shirts," changes their search to "denim shirts," then changes their search again to "button-down shirts," only one MAU would be counted because it is based on a unique user ID. This is in contrast to impressions, which would have counted three impressions, making the total usage count now 11 impressions.
Verify MAUs with the Experimentation Events Export
Optimizely Experimentation calculates MAUs using the server timestamp, as opposed to the timestamp on the client device where the MAU originated. This makes it possible to verify MAUs accurately down to the experiment level.
You can use Optimizely Experimentation's Experimentation Events Export to get a complete list of all MAUs that occurred within a specific time period. You can then compare that information to your invoice or determine whether any of your experiments are generating more MAUs than they should be. To learn how to access that data, see Experimentation Events Export services in Optimizely Experimentation.Experimentation Events Export services in Optimizely Experimentation.
View MAU usage
Note
See View Impression usage if you are billed by impressions.
To view your Monthly Usage Information in Optimizely Experimentation, go to Account Settings > Plan.
Note
You must be an administrator in Web Experimentationadministrator in Web Experimentation or an administrator in Feature Experimentationadministrator in Feature Experimentation to access the Plan tab.
Note
If you are an administrator in Opti ID, you can access a more detailed view of your Optimizely Experimentation usage data on the usage and billing dashboards in the Opti ID Admin Centerusage and billing dashboards in the Opti ID Admin Center.
Forecast MAUs
To forecast MAUs, ask the following questions:
- Where will you be using Optimizely Experimentation? These could be multiple channels, regions, business units, and so on (for example, Android, iOS, website, and so on).
- How many unique users/visitors do you have per month? Include unique users/visitors, not page views. Excludes bot traffic. We recommend you get these numbers from internal analytics. Note: Provide the average across the year, note the peak.
- How complete is your Optimizely Experimentation implementation? If the Feature Experimentation SDK or Web snippet runs for most users (for example, feature flags in-app navigation or experiments on a product detail page layout), this should be 90-100%. If Optimizely Experimentation is only used on a small part of the site (for example, personalizing a few landing pages, rolling out minor features only) this could be as low as 5-10%.
- How much annual growth do you expect? Year over year, how much more traffic do you expect?
Note
MAUs are a shared pool between Optimizely Web Experimentation and Optimizely Feature Experimentation products and roll over the next month if not used.