Read the Optimizely Experimentation log

31 min

Optimizely Web Experimentation

Optimizely Personalization

The Optimizely Web Experimentation log is a valuable resource. It lets you "read Optimizely Web Experimentation's mind" by printing the execution of targeting and activation decisions, variation changes, events, and third-party integrations on a page in your browser's console. Use the Optimizely Web Experimentation log to investigate issues you cannot easily diagnose.

The log can help you check the following:

If an experiment or campaign is loading correctly.

If you qualify for an audience condition.

If changes you make on a page are applied.

Start by enabling the log with a query parameter or the JavaScript API.

Note

Are you looking for more technical resources? Find out more about the logmore about the log in the developer documentation.

Enable the log with a query parameter

You can enable the log by adding a query parameter before you load the page:

Add the query parameter optimizely_log=info.

Reload the page to read the log.

In Optimizely Web Experimentation, you must append the optimizely_log parameter and reload the page to see the log. This minimizes the logging overhead for visitors and pageviews that don't request the log.

Enable the log with the JavaScript API

Open your browser's developer consoledeveloper console.

Use the JavaScript APIJavaScript API to show messages from the page load. Append the query parameter and push the log before the snippet loads to see information from the current page load.

window['optimizely'] = window['optimizely'] || [];
window['optimizely'].push({ type: 'log', level: 'info' });

Other JavaScript on the page may have produced console output as the page was loading, so you must find where the Optimizely Web Experimentation log starts. Search for a line that starts with, Setting log level to...

To modify the verbosity of the messages, replace info with one of the levels below (listed from least to most verbose):

off – No messages.

error – Only errors (in other words, unexpected conditions that might cause the snippet to run improperly).

warn – Unusual conditions that might indicate a misconfiguration.

info – Recommended if you are trying to identify what is running and what is not.

debug – Recommended if you are trying to identify why something is or is not running.

Troubleshoot with the log

Messages are printed in the log in the same chronological order as the occurring events. If you enable the log with a query parameter, append the query parameter before you load the page to see the log.

Log messages have the following format.

  461 | Optly / Page / Failed to match page for "Marketing: Home Page" (47277070544)
  > Array[2]

461 – Indicates the time at which the message was captured, in milliseconds, relative to the time at which the snippet started to execute.

Optly – Indicates that the message is from the Optimizely Web Experimentation snippet:

Page – (Optional) indicates the general class of behavior that the log message describes. In the Optimizely example, the message is about pagepage evaluation. If the optional third part is missing from the message, the message corresponds to the "Main" routine of the snippet.

Failed to match page for "Marketing: Home Page" (47277070544) – The remainder is the actual text of the message.

Array[2] – This message may include references to JavaScript objects you can inspect using the console. In this case, the included array encodes the page URL targetingpage URL targeting conditions.

Use the following sections to troubleshoot pages, campaigns, audience, traffic allocation and bucketing, and variation code with the Optimizely Web Experimentation log. Work through each section according to the Optimizely Web Experimentation order of activationorder of activation.

Sections include a general log format, example log, and search string.

The general log format shows which information the log message usually includes.

The example log is an example of a real log message you might see.

The search string helps you find the log message for each troubleshooting step.

Note

You may see the word Null instead of Page, Campaign, Variation, Experiment, or Audience Name in your log message. This means the privacy setting of Mask descriptive names in project code and third-party integrations is enabled. If this setting is enabled, use API NamesAPI Names to determine which IDs Optimizely Web Experimentation refers to.

Before troubleshooting, enable the log with a query parameter or the JavaScript API and reload the page. If you use API Names to identify masked IDs, open the API NamesAPI Names settings for the campaign you are troubleshooting in another browser window.

Troubleshoot pages

Pages must activate for experiments to activate.

Are pages active?

Check whether pages are active. If pages are active, the URL you are troubleshooting met the URL Targeting or activation conditions.

General log format

{time} | Optly / Activated Page "{page name}" ({page ID})

Example log

134 | Optly / Activated Page "Home" (3536920000)

Search string

Activated Page

Pages failed to activate

Check for pages that failed to activate. If a page (or pages) failed to activate, the URL you are troubleshooting failed the URL targeting or activation conditions.

General log format

{time} | Optly / Page / Failed to activate view for "{page name}" ({page ID})

Example log

216 | Optly / Failed to activate view for "Home" (3536920000)

Search string

Failed to activate view

Troubleshoot campaigns

Campaigns do not display in the log if they are not targeted to the page you are troubleshooting. To check which campaigns are activated for the page:

General log format

{time} | Optly / Deciding Campaigns for Page "{page name}" ({page ID})

Example log

149 | Optly / Deciding Campaigns for Page "Home" (3536920208) : 
  "Homepage Test" (7729960575),
  "Second Homepage Test" (8163204973),
  "Third Homepage Test" (8169565973)

Search string

Optly / Deciding Campaigns for Page

Search for executing actions and expand the object to find different changeSet objects, including whether custom code has run on the page. Executing actions also includes what selector was used for the element and whether the attribute or CSS of the element was changed.

Troubleshoot audiences

Does the visitor meet the audience conditions?

If a visitor does not meet the audience conditions, the log shows the visitor is in an empty object ([ ]), which represents the everyone audience condition.

General log format

{time} | Optly / When deciding Campaign "{campaign name}" ({campaign ID})
  visitor is in audiences: [ ]
  and not in audiences: ["{Audience Condition 1}","{Audience Condition 2}"]

Example log

138 | Optly / When deciding Campaign "My Campaign" (7729960575) 
  visitor is in audiences: [ ]
  and not in audiences: ["Audience Condition 1","Audience Condition 2"]

Search string

When deciding Campaign

Does the visitor meet one but not both audience conditions?

A visitor may meet one but not both audience conditions.

General log format

{time} | Optly / When deciding Campaign "{campaign name}" ({campaign ID})
  visitor is in audiences: ["{Audience Condition 1}", Object]
  and not in audiences: ["{Audience Condition 2}", Object]

Example log

138 | Optly / When deciding Campaign "My Campaign" (7729960575)
  visitor is in audiences: ["Audience Condition 1", Object]
  and not in audiences: ["Audience Condition 2", Object]

Does the visitor meet all audience conditions?

If a visitor may meet all audience conditions, the log shows the visitor is not in an empty object ([ ]).

General log format

{time} | Optly / When deciding Campaign "{campaign name}" ({campaign ID})
  visitor is in audiences: ["{Audience Condition 1}","{Audience Condition 2}"]
  and not in audiences: [ ]

Example log

193 | Optly / When deciding Campaign "My Campaign" (7729960575)
  visitor is in audiences: ["Audience Condition 1","Audience Condition 2"]
  and not in audiences: [ ]

Troubleshoot traffic allocation and bucketing

This section helps you identify which campaign or experiment and variation a visitor was bucketedbucketed into.

General log format

{time} | Optly / Recording decision for Campaign "{campaign name}" ({campaign ID}) 
  Object {bucketingId: "{x}", 
          audienceIds: Array[{x}], 
          globalHoldback: {x}, 
          preferredVariationMap: Object, 
          activationId: "{x"...} -> 
  Object{layerId: "{x}", 
         experimentId: "{experiment ID}", 
         variationId: "variation id",
         isLayerHoldback: {true or false}}

Example log

157 | Optly / Recording decision for Campaign "Homepage Test" (11111111111) 
  Object {bucketingId: "oeu100000000.1111111111", 
          audienceIds: Array[0], 
          globalHoldback: 0, 
          preferredVariationMap: 
          Object, activationId: "e2240b20-6531-4824-a111-9d7ccfc2cc00"…} -> 
  Object {layerId: "9999999", 
          experimentId: "11111111", 
          variationId: "2222222", 
          isLayerHoldback: false}

Search string

Recording decision

To confirm traffic allocation and bucketing for a visitor, look for the values for the following elements in the log message:

layerId – The campaign or experiment the visitor was bucketed into.

variationId – The variation the visitor was bucketed into.

isLayerHoldback – If true, the visitor was not bucketed into the campaign or experiment, even if a variation ID was assigned.

Troubleshoot variation code

This section helps you learn which variation code is being evaluated on the page. Variation code can be synchronous or asynchronoussynchronous or asynchronous. Optimizely Web Experimentation applies campaign or experiment variation code immediately after the traffic allocation decision if the timing of the changes is set to synchronous, which is the default setting.

Synchronous variation code

Synchronous is loaded at the time the snippet loads.

General log format

{time} | Optly / Executing actions for Campaign "{campaign name}" ({campaign ID})
  Page "{page name}" ({page ID})

Example log

283 | Optly / Executing actions for Campaign "Homepage test" (7729960575)
  Page "Home" (3536920208)

Search string

Executing actions

Asynchronous variation code

Asynchronous variation code is loaded after the snippet:

General log format

{time} | Optly / Executing "load" with arguments: Object

Example log

2059| Optly / API / Executing "load" with arguments: Object

Search string

"load" with arguments

Visitor is not eligible for the A/B test or Campaign

General log format

{time} | Optly / NOT applying Variation for Campaign "{campaign name}" ({campaign ID})
  ({reason})

Example log

182 | Optly / NOT applying Variation for Campaign "Homepage test" (818060000)
  (visitor is not eligible for any Experiments)

Search string

NOT applying

Visitor was bucketed into the holdback

General log format

{time} | Optly / NOT applying Variation for Campaign "{campaign name}" ({campaign ID})
  ({reason})

Example log

182 | Optly / NOT applying Variation for Campaign "Holdback experiment" (818060000)
  (visitor is in layer holdback)

Search string

NOT applying Variation

Find errors with variation code

General log format

{time} | Optly / Action / Failed to apply change:
  {{Where the error is coming from)}
  {(Reason for the error)}

Example log

182 | Optly / Action / Failed to apply change:
  {dependencies: Array(0), 
    type: "custom_code", 
    id: "0282ED9C-A0E8-40AE-9C62-0EF40009FCDA", 
    value: ƒ}
  n {stack: "ExecError: ReferenceError: myfunction is not define…

Search string

Failed to apply change