Review impression usage

This article describes how impressions work at Optimizely Experimentation, similar to that used by online media, but the nature of our technology introduces some key differences.

In Optimizely Web Experimentation, an impression is counted for every Optimizely Web Experimentation pagOptimizely Web Experimentation page each time a visitor sees an Optimizely Web Experimentation experience as the result of a test or personalization campaign.

Impressions are the unit of measurement for usage in Optimizely Feature Experimentation and Optimizely Web Experimentation, but Optimizely Web Experimentation experiments include an extra layer: an experiment in Optimizely Web Experimentation can run on multiple Optimizely Web Experimentation pagesOptimizely Web Experimentation pages. Every time a visitor activates a page within an Optimizely Web Experimentation experiment, Optimizely Web Experimentation counts an impression.

Sometimes, an experiment may have two pages that target the same URL (this is common for single-page applicationssingle-page applications). When a visitor activates both pages during the same visit, this counts as two impressions.

Impressions are not counted for visitors bucketed into the holdback:

A decision request is sent each time an Optimizely Web Experimentation experiment is activated. Decision requests look like this:

In the request payload, the decision attribute indicates the experiment that it applies to.

The following scenario has three multipliers:

The Attic and Button company is experimenting on www.atticandbutton.us. Consider a visitor who starts by visiting the Attic and Button homepage, where there are three experiments running. One of these experiments has two Optimizely Experimentation pagesOptimizely Experimentation pages that target the homepage:

Experiment 1

Experiment 2

Experiment 3

The visit counts as four impressions because four page activations occur when the visitor views the homepage. Impressions are shown in green below.

If the visitor refreshes the page, another four impressions are counted. Attic and Button’s account usage now totals eight impressions.

Now, suppose you are also running a search algorithm experiment with Optimizely Feature Experimentation on the homepage. When a visitor types a search term, the results are refreshed without reloading the page. The Optimizely Feature Experimentation SDK makes a decision for a variation every time a search is done. This means that if a visitor searches for "shirts," changes their search to "denim shirts," then changes their search again to "button down shirts," another three impressions are counted. The total usage count is now 11 impressions.

See the developer documentation for information on when impressions and decisionsimpressions and decisions are counted for Optimizely Feature Experimentation.

In Optimizely Full Stack Experimentation, an impression is counted each time an experiment is activated and a decision event is sent:

Impressions are only counted for visitors who are bucketed into a variation in an experiment. The activate() call alone does not generate an impression.

Let us say that you are running an experiment on your homepage. In theory, every visitor to your homepage triggers the activate() call. However, before an impression is created, the activate() method does two things:

If the visitor meets the audience conditions, the activate() call assigns a variation for the visitor. An impression is counted only for visitors receiving one variation assignment.

If the visitor does not meet the audience conditions, the activate() call assigns a nil or NULL variation for the visitor, depending on the language. An impression is not counted for visitors who receive a nil or NULL variation assignment means these visitors do not count against your allotted number of impressions.

All rollouts are excluded from impression counts.

Optimizely Experimentation uses the server timestamp to calculate impressions instead of the timestamp on the client device where the impression originated. Doing so lets you accurately verify impressions down to the experiment level.

You can use Optimizely Experimentation's Experimentation Events Export to get a complete list of monthly active users within a specific period. You can then compare that information to your invoice or determine whether your experiments generate more monthly active users than they should. To learn how to access that data, see Experimentation Events Export.Experimentation Events Export.

On June 23, 2020, Optimizely Experimentation began updating impression logic which is in effect automatically for your accounts. The logic introduces fixed-interval bucketing, which de-duplicates impressions received within fixed 5-second intervals for every user in an experiment. This de-duplicate is based on the received timestamp of the event. This de-duplication process is done by Optimizely Experimentation and does not require any action on your part to receive the benefit. 

If you are on Impressions billing, to view your Impression Usage Summary in the Optimizely Application, go to Account Settings > Usage.