Handle user IDs in Feature Experimentation
- Optimizely Feature Experimentation
Optimizely Feature Experimentation identifies visitors with user IDs and determines which variation they see in an experiment (A/B testA/B test or multi-armed banditmulti-armed bandit) or a flag delivery (targeted delivery)flag delivery (targeted delivery). Feature Experimentation lets you define user IDs based on your testing strategy. For example, you can use a first-party cookie, a device ID for anonymous users, or a stored identifier for logged-in users.
Consistent user IDs prevent users from being potentially re-bucketed into different variations, ensuring reliable experiment results. If your tests span multiple devices or applications, you can use a UUID to maintain a consistent experience for each user. Alternatively, you can use bucketing IDsbucketing IDs to provide a consistent experience across different identifiers.
Tips for user IDs
- Ensure user IDs are unique – Feature Experimentation buckets users based on the user IDs you provide, so ensure these IDs are unique within your testing population.
- Anonymize user IDs – Optimizely processes user IDs exactly as provided. Anonymize any personally identifiable information (PII), such as email addresses, according to your company's policies.
- Use one namespace per project – Feature Experimentation assumes a single namespace for user IDs per project. Mixing different ID conventions within a project (for example, anonymous visitor IDs and UUIDs) can affect mutual exclusivitymutual exclusivity between tests.
- Use logged-out or logged-in IDs – Optimizely does not alias logged-out IDs with logged-in IDs. Persist logged-out IDs for the duration of a test if you need to track users across authentication states. See the Handle multiple user IDs section.
- iOS 14 ID for advertisers and disclosure updates – Feature Experimentation SDKs do not access user data or identifiers that you do not explicitly pass in through instrumented API calls. As a result, you should not depend on SDK changes planned to accommodate Apple's changes to its privacy disclosure and opt-in policy related to their ID for Advertisers (IDFA). You should review Apple's full requirements and guidance, other identifier options, and your data use carefully to determine the correct path forward for your application and usage.
Create a user context example
Feature Experimentation SDKs manage user IDs through user contexts. When you create a user context, you define a user ID and optional attributes for audience targeting. The SDKs consistently assign a user to the same variation as long as the traffic distribution is not modified. See Why you should not change a running experimentWhy you should not change a running experiment.
For SDK-specific code examples, see the Create a user context example sectionCreate a user context example section in the developer documentation.
For information, see the SDK's user context documentation.
- AndroidAndroid
- C#C#
- FlutterFlutter
- GoGo
- JavaJava
- JavaScriptJavaScript – SDK versions 6.0.0 and later.
- JavaScript (Browser)JavaScript (Browser) – SDK versions 5.3.5 and earlier.
- JavaScript (Node)JavaScript (Node) – SDK versions 5.3.5 and earlier.
- PHPPHP
- PythonPython
- RubyRuby
- SwiftSwift
Handle multiple user IDs
If a user interacts with an experiment or targeted delivery while logged out and then logs in, Optimizely does not merge their logged-out identity with their logged-in identity. You should persist the logged-out ID and continue using it after login instead of generating a new one if you need to track a user's journey across login states (for example, seeing the same experiment variation before and after login).
Risks to changing a logged-out ID
- Inconsistent user experience – Users may see different variations before and after logging in.
- Corrupted experiment data – Optimizely may attribute conversion events to different IDs, inflating visitor counts and skewing results.
- Billing inefficiencies – If you are on the Monthly Active User (MAU)-based pricing model, unnecessary user ID changes can increase your MAUs. See the users and invoices section.
Best practices for multiple user IDs
- Do not change user IDs in an existing user context, as it may alter variation assignments.
- Maintain separate user context instances for anonymous and logged-in users.
- Only use the logged-in user context where necessary, specifically in omnichannel experiments.
- Ensure event tracking is performed separately for each user context to maintain accurate results.
Recommended approach for handling multiple user IDs
To maintain experiment consistency, create separate user context instances for anonymous and logged-in users. Never modify the user ID in an existing user context. The code sample demonstrating how to do this are available in the developer documentation developer documentation.
Users and invoices
Starting in September 2020, Optimizely Feature Experimentation offers monthly active user (MAU)monthly active user (MAU) pricing as an alternative to impressions pricing. Optimizely counts an MAU each time one of the following occurs for a unique user ID:
- When you call a Decide method – A decision event (impression) is triggered.
- When you call a Track Event method – A conversion event is triggered.
The MAU count includes unique anonymous IDs. If you use Optimizely Web ExperimentationOptimizely Web Experimentation and Optimizely Feature Experimentation, you can override anonymous Optimizely Web Experimentation user IDs with known Feature Experimentation user IDs to avoid overcounting. See Bring your own IDBring your own ID.