Test Commerce Search v3

17 min

After you implement Commerce Search v3 for Optimizely Configured Commerce, you can test the default controls to see if you need to edit any settings.

Note

To ensure that user events effectively influence search results, the Google Cloud Vertex AI Search must reach a certain threshold of total user events ingested into its Retail Search project. This threshold is necessary for the AI to be adequately trained on the events and start influencing the search results, thereby providing more relevant results. During the initial phase, the AI may not immediately reflect the impact of user events on search results until sufficient data is collected and processed.

Monitor user events flow

Ensure that the Daily User Events Sync integration job runs successfully. This guarantees that user events sent from the Commerce storefront and backend to Optimizely Data Platform (ODP) are synced to Google Cloud Vertex AI Search (Retail Search).

Rebuild index

Go to Marketing > Indexing in the Admin Console and click Rebuild All.

Check the Indexing Jobs log to monitor the progress of product catalog ingestion into Retail Search. Review the logs for any error messages related to products that failed to index and examine all error and warning messages.

Update the product details to correct any errors, then rerun the Rebuild Index job.

Ensure the process completes successfully. Report any unresolved indexing errors or warnings to Optimizely Support.

(For multiple languages and websites) Verify that the job successfully indexed each website's product catalog to Retail Search for all active website languages.

You should rebuild a couple of times to sync product attribute configuration and searchability.

Common indexing errors

The indexing logs most often report these errors and warnings. Correct the product data, then rerun the indexing job.

Error	Resolution
product.priceInfo.originalPrice is required when product.priceInfo.priceEffectiveTime is set.	The product has a Sale Start Date but no List Price. Set a list price, or remove the sale start date.
product.priceInfo.price (50) is greater than product.priceInfo.originalPrice (25), but it should be less than or equal to it.	The product's sale price is higher than its list price. Correct the Sale Price so it is less than or equal to the List Price.
product.attributes.value.text contains 401 values, but it allows a maximum of 400 values.	A single product can have at most 400 values for the same attribute type. Products normally have one value per attribute type, so this error usually comes from a system attribute. For example, a product assigned to a large number of categories and subcategories can exceed 400 category ID values. Reduce the number of values.
Field product.priceInfo.price must be nonnegative, but was -5467.	The product has a negative Sale Price or List Price. Correct the price.
product.attributes.value.text cannot be empty or contain only whitespaces, control characters or non-characters.	An attribute value contains control characters or non-characters. Clean up the attribute value.
The attribute value for {attribute name} exceeds the max length of 256 unicode characters. It will be trimmed to 256 characters.	Commerce Search v3 limits product attribute values, and some system attribute values, to 256 characters. Keep values under 256 characters when you use them for search and filtering.
Identity, en-US Google Retail Search - Unable to update Dynamic Facet Spec.	The Identity website is an internal website and is not configured for Commerce Search v3. To stop these errors, go to the Identity website's languages and set Is Live to off for the en-US language.

Monitor how user events influence search results

Over time, verify that user events (such as searching or browsing products, adding products to the cart, purchases, viewing product details, and so on) influence the Google Vertex AI search engine to return more relevant search results.

Log in to the storefront and assign a customer ship-to and bill-to. Use this customer context throughout these steps. The user can be anonymous when logged out, but all events triggered are associated with the login afterward.

Perform searches on the storefront as defined in Test search.

Perform the following actions on the storefront that would track and send user events:

Add products to cart.

View product details.

View the cart.

Search products and browse categories.

Sort products on the Product List page.

Filter products on the Product List page.

Select a search suggestion from the autocomplete results in the search bar.

Verify that products are ranked higher in search results based on past clicks.

Verify that products with higher purchase rates are ranked higher in search results.

Verify that products with higher user interactions (such as adding to cart and viewing product details) are ranked higher in search results.

Verify that recent user events influence search results.

Verify that long-term user events have a lasting influence on search results.

Verify that products with negative user events (such as no detail view and not adding to cart) are ranked lower in search results.

Test text search and browse search

Use the following list to help test your searches and browses after successfully rebuilding the index:

Browse through the category menu and view products from different categories. Review how the products are listed by their relevancy on the Product List page. Evaluate how Retail Search AI intelligently orders the products and provide any useful feedback on this relevance.

Go to the Brands pages and review how the products are listed by their relevance.

Search using partial terms, words, prefixes, phrases, full keywords/terms, and natural language that may potentially match any of the following product fields:

Title

Description

Part Numbers (like ERP, Manufacturer Item Number, and Model Numbers)

Attributes (like color and size)

Any other searchable product fields

Select and deselect filters on the Product List page and review how the products are listed and sorted by the best relevancy determined by Retail Search AI.

Sort products using different available sort order options and review how the products are sorted. Review section Semantic Embedded Based Filtering affecting search results when sorting by default and non-default sort option.

Apply different pagination size options and go through the subsequent pages.

Test how the Retail Search AI suggests spelling corrections when searching with terms that may be misspelled or when a correct term might potentially match products in the catalog. Test how Configured Commerce does not suggest corrections to search terms that are part numbers when Exclude part number from autocorrect is turned On.

Test searches with the Search Query Expansion setting turned off or on. Perform the previous steps and review how the products are listed by relevancy by Retail Search AI.

Query expansion increases recall for query terms with few results, especially long-tail queries. For example, searching Google Pixel 5 without query expansion only returns google_pixel_5. With query expansion, you may also get google_pixel_4a_with_5g, google_pixel_4a, and google_pixel_5_case.

When a shopper uses an ambiguous or multi-word search phrase, they may get an empty response. With query expansion turned on, the request is analyzed, and the product list is expanded based on the search query.

Search on a specific website or language

Website-specific catalog indexing

Each website has its catalog indexed to its associated Retail Search projects by website and language.

The search results should only include products from the specific website being searched.

Language-specific search

Ensure that the AI understands the context of language-specific searches.

Ensure that the search functionality is consistent across different languages.

Autocomplete

Go to the storefront and test the autocomplete. Enter different search terms in the search bar and wait for any autocomplete suggestions to display.

The following results are expected to display based on the search term. You may not see most suggestions until the autocomplete model trains on events.

Search - Suggestions

Categories - Popular

Categories - Suggestions

Brands - Popular

Brands - Suggestions

Autocomplete triggers after entering two characters. Test the following scenarios:

Common search terms (like shoes or laptop)

Synonyms (like sofa and couch)

Related terms (like phone and charger)

Brands (like nike and reebok)

Categories (like grinders and faucets).

You should also verify suggestions for different languages on various websites by entering search terms in different languages (like English, Spanish, and French).

Requirements for suggestions to display

Autocomplete suggestions are generated based on search events and require the following:

Minimum search activity – The system must collect enough user search behavior to train the model.

Auto-learning enabled – Autocomplete relies on a machine learning model that updates daily and uses up to 180 days of search history.

Zero-result filtering – Queries that consistently return no results are excluded.

Daily refresh cycle – New data takes about two days to fully propagate into suggestions.

Rate-limit handling

Verify how Commerce Search v3 handles rate limiting from the Commerce Search Service.

Confirm the following behavior when the Commerce Search Service returns an HTTP 429 response:

The Autocomplete API returns an empty result set.

The Search API falls back to Elasticsearch.

The system writes a warning message to the log once per minute for the duration of the rate-limit window.

Facets

Two types of facets, or filters, display on the Commerce storefront Search Results page:

Static system filters – Categories, Brands, Product Lines, Price, and Stocked Items.

Product attribute filters – product attribute types, custom product properties, and custom product fields.

The Enable Dynamic Facets setting controls whether product attribute filters display on the Search Results page. Changing this setting requires a full index rebuild.

Dynamic facets

Dynamic facets are product attribute filters that the search system generates automatically. Commerce Search v3 ranks them by user interactions and by the relevance of the search results. It generates these facets with AI so shoppers can refine their results.

Note

Attribute type filters are the only ones dynamically generated by Retail Search. Filters such as Categories, Brands, Stocked Items, Price, and Product Lines are all system filters and are not dynamically generated.

How the Enable Dynamic Facets setting affects filters

Page	Enable Dynamic Facets off	Enable Dynamic Facets on
Search results	Static system filters display. No attribute filters display.	Static system filters display, along with AI-generated attribute filters.
Category browse	Static system filters display, except the Categories filter. Up to 200 attribute filters assigned to the category display.	Static system filters display, except the Categories filter. AI-generated attribute filters display.

On category browse pages with Enable Dynamic Facets off, attribute filters display only when Attribute Filters is also On. The 200-filter cap applies to the active attribute types assigned to the category, in sort order.

In each case, custom extensions can display specific forced facets. The facet limits still apply. See Limitations of Commerce Search v3.

Dynamic facets cold start

A newly migrated catalog has no AI training data, so dynamic facets take time to display. Google recommends running Commerce Search v3 in shadow mode before you make it the active search provider. User events then accumulate in the background. See Shadow mode in Enable Commerce Search v3 in production.

Test dynamic facets

Observe the dynamic facets that may display in the following conditions:

When searching with different search terms.

When browsing by category.

When browsing by brand.

When filtering or sorting product results.

Semantic embedded-based filtering

With default sorting, Best Match, Retail Search displays a range of search results, including products that are popular or trending, even if they are slightly relevant. With non-default sorting, like Product: A to Z, some products may display higher in the results even if they are not the most relevant to the search query. To improve the quality of non-default search results, Retail Search uses semantic embedding-based filtering. This technique helps to filter out less relevant items, ensuring that the search results are more relevant to the user's query. While this filtering improves the relevance of search results, it may also reduce the total number of search results.

There may be instances where sorting by a non-default sort option, such as Product: A to Z, results in no products being displayed. This is expected behavior.

Part number search

When validating how Commerce Search v3 handles product identifiers, include tests that use partial text for all fields that contain part‑number‑type data. This includes product numbers, manufacturer item numbers, model numbers, and other structured identifiers. Test with prefixes, mid‑string fragments, and truncated endings to confirm that Google Cloud Retail Search can interpret incomplete inputs and still return the correct products.

To support partial text matching for part numbers, you must enable N‑gram indexing for any fields that store part‑number‑type data. N‑gram indexing breaks long identifiers into smaller searchable segments, letting Google Cloud Retail Search match products, even when only part of the number is provided.

Go to Settings > Enable Ngram under Google Cloud Retail Search.

You can also enable N‑gram indexing on other product identifiers. Use Product Data Fields to specify which fields should support partial‑text search. For example, to make a product field or a custom field searchable using partial text, enter modelNumber as the Product Data Field. If the identifier is a product attribute, such as vendorNumber, enter vendorNumber in the Product Data Fields list.

After you performing a rebuild, you must update the newly added data fields in Marketing > Commerce Search v3 > Attribute Controls with the following settings for all applicable websites and languages, followed by another rebuild to apply the changes:

Dynamic Facetable – False

Exact Match – True

Searchable – True

After you enable N‑grams and configure the appropriate fields, reindex the catalog a couple of times so the updated N-gram tokenization applies to all relevant fields.

Verify canonical filter

Test how the Canonical filter setting influences query expansion behavior.

Enable Search Query Expansion and Canonical filter in the Admin Console. Perform searches using long-tail queries that typically return few or no results.

Compare results with the Canonical filter setting turned on and off:

Verify that query expansion triggers when the original query returns few results.

Verify that applied filters (such as category, brand, or attribute filters) do not incorrectly prevent query expansion when Canonical filter is enabled.

Verify that query expansion evaluates results against a broader catalog view rather than only the filtered result set.

Apply filters, sorting, and pagination, and confirm the following:

Filtering does not prevent relevant expanded results from appearing when appropriate.

Sorting does not negatively affect the relevance of expanded results.

Pagination returns consistent and expected results across pages.

Verify that the final search results remain unchanged in structure:

Confirm that Canonical filter does not directly modify the products returned.

Confirm that the setting only affects when query expansion is triggered.

Browse categories and confirm that:

Category navigation behavior is not affected by the Canonical filter setting.