Test Commerce Search v3
After you implement Commerce Search v3 for Optimizely Configured Commerce, you can test the default controls to see if you need to edit any settings.
Note
To ensure that user events effectively influence search results, the Google Cloud Vertex AI Search must reach a certain threshold of total user events ingested into its Retail Search project. This threshold is necessary for the AI to be adequately trained on the events and start influencing the search results, thereby providing more relevant results. During the initial phase, the AI may not immediately reflect the impact of user events on search results until sufficient data is collected and processed.
Monitor user events flow
Ensure that the Daily User Events Sync integration job runs successfully. This guarantees that user events sent from the Commerce storefront and backend to Optimizely Data Platform (ODP) are synced to Google Cloud Vertex AI Search (Retail Search).
Rebuild index
- Go to Marketing > Indexing in the Admin Console and click Rebuild All.
- Check the Indexing Jobs log to monitor the progress of product catalog ingestion into Retail Search. Review the logs for any error messages related to products that failed to index and examine all error and warning messages.
- Update the product details to correct any errors, then rerun the Rebuild Index job.
- Ensure the process completes successfully. Report any unresolved indexing errors or warnings to Optimizely Support.
- (For multiple languages and websites) Verify that the job successfully indexed each website's product catalog to Retail Search for all active website languages.
You should rebuild a couple of times to sync product attribute configuration and searchability.
Monitor how user events influence search results
Over time, verify that user events (such as searching or browsing products, adding products to the cart, purchases, viewing product details, and so on) influence the Google Vertex AI search engine to return more relevant search results.
- Log in to the storefront and assign a customer ship-to and bill-to. Use this customer context throughout these steps. The user can be anonymous when logged out, but all events triggered are associated with the login afterward.
- Perform searches on the storefront as defined in Test search.
- Perform the following actions on the storefront that would track and send user events:
- Add products to cart.
- View product details.
- View the cart.
- Search products and browse categories.
- Sort products on the Product List page.
- Filter products on the Product List page.
- Select a search suggestion from the autocomplete results in the search bar.
- Verify that products are ranked higher in search results based on past clicks.
- Verify that products with higher purchase rates are ranked higher in search results.
- Verify that products with higher user interactions (such as adding to cart and viewing product details) are ranked higher in search results.
- Verify that recent user events influence search results.
- Verify that long-term user events have a lasting influence on search results.
- Verify that products with negative user events (such as no detail view and not adding to cart) are ranked lower in search results.
Test text search and browse search
Use the following list to help test your searches and browses after successfully rebuilding the index:
- Browse through the category menu and view products from different categories. Review how the products are listed by their relevancy on the Product List page. Evaluate how Retail Search AI intelligently orders the products and provide any useful feedback on this relevance.
- Go to the Brands pages and review how the products are listed by their relevance.
- Search using partial terms, words, prefixes, phrases, full keywords/terms, and natural language that may potentially match any of the following product fields:
- Title
- Description
- Part Numbers (like ERP, Manufacturer Item Number, and Model Numbers)
- Attributes (like color and size)
- Any other searchable product fields
- Select and deselect filters on the Product List page and review how the products are listed and sorted by the best relevancy determined by Retail Search AI.
- Sort products using different available sort order options and review how the products are sorted. Review section Semantic Embedded Based Filtering affecting search results when sorting by default and non-default sort option.
- Apply different pagination size options and go through the subsequent pages.
- Test how the Retail Search AI suggests spelling corrections when searching with terms that may be misspelled or when a correct term might potentially match products in the catalog. Test how Configured Commerce does not suggest corrections to search terms that are part numbers when Exclude part number from autocorrect is turned On.
- Test searches with the Search Query Expansion setting turned off or on. Perform the previous steps and review how the products are listed by relevancy by Retail Search AI.
- Query expansion increases recall for query terms with few results, especially long-tail queries. For example, searching Google Pixel 5 without query expansion only returns google_pixel_5. With query expansion, you may also get google_pixel_4a_with_5g, google_pixel_4a, and google_pixel_5_case.
- When a shopper uses an ambiguous or multi-word search phrase, they may get an empty response. With query expansion turned on, the request is analyzed, and the product list is expanded based on the search query.
Search on a specific website or language
- Website-specific catalog indexing
- Each website has its catalog indexed to its associated Retail Search projects by website and language.
- The search results should only include products from the specific website being searched.
- Language-specific search
- Ensure that the AI understands the context of language-specific searches.
- Ensure that the search functionality is consistent across different languages.
Autocomplete
Go to the storefront and test the autocomplete. Enter different search terms in the search bar and wait for any autocomplete suggestions to display.
The following results are expected to display based on the search term. You may not see most suggestions until the autocomplete model trains on events.
- Search - Suggestions
- Categories - Popular
- Categories - Suggestions
- Brands - Popular
- Brands - Suggestions
Autocomplete triggers after entering two characters. Test the following scenarios:
- Common search terms (like shoes or laptop)
- Synonyms (like sofa and couch)
- Related terms (like phone and charger)
- Brands (like nike and reebok)
- Categories (like grinders and faucets).
You should also verify suggestions for different languages on various websites by entering search terms in different languages (like English, Spanish, and French).
Requirements for suggestions to display
Autocomplete suggestions are generated based on search eventssearch events and require the following:
- Minimum search activity – The system must collect enough user search behavior to train the model.
- Auto-learning enabled – Autocomplete relies on a machine learning model that updates daily and uses up to 180 days of search history.
- Zero-result filtering – Queries that consistently return no results are excluded.
- Daily refresh cycle – New data takes about two days to fully propagate into suggestions.
Rate-limit handling
Verify how Commerce Search v3 handles rate limiting from the Commerce Search Service.
Confirm the following behavior when the Commerce Search Service returns an HTTP 429 response:
- The Autocomplete API returns an empty result set.
- The Search API falls back to Elasticsearch.
- The system writes a warning message to the log once per minute for the duration of the rate-limit window.
Dynamic facets
Dynamic facets are search filters, specifically attribute type filters, that are automatically generated and ranked by the search system based on user interactions and the relevance of the search results. These facets are dynamically generated by AI to help users refine their search results more effectively.
Note
Attribute type filters are the only ones dynamically generated by Retail Search. Filters such as Categories, Brands, Stocked Items, Price, and Product Lines are all system filters and are not dynamically generated.
Observe the dynamic facets that may display in the following conditions:
- When searching with different search terms.
- When browsing by category.
- When browsing by brand.
- When filtering or sorting product results.
Semantic embedded-based filtering
With default sorting, Best Match, Retail Search displays a range of search results, including products that are popular or trending, even if they are slightly relevant. With non-default sorting, like Product: A to Z, some products may display higher in the results even if they are not the most relevant to the search query. To improve the quality of non-default search results, Retail Search uses semantic embedding-based filtering. This technique helps to filter out less relevant items, ensuring that the search results are more relevant to the user's query. While this filtering improves the relevance of search results, it may also reduce the total number of search results.
There may be instances where sorting by a non-default sort option, such as Product: A to Z, results in no products being displayed. This is expected behavior.
Part number search
When validating how Commerce Search v3 handles product identifiers, include tests that use partial text for all fields that contain part‑number‑type data. This includes product numbers, manufacturer item numbers, model numbers, and other structured identifiers. Test with prefixes, mid‑string fragments, and truncated endings to confirm that Google Cloud Retail Search can interpret incomplete inputs and still return the correct products.
To support partial text matching for part numbers, you must enable N‑gram indexing for any fields that store part‑number‑type data. N‑gram indexing breaks long identifiers into smaller searchable segments, letting Google Cloud Retail Search match products, even when only part of the number is provided.
Go to Settings > Enable Ngram under Google Cloud Retail Search.
You can also enable N‑gram indexing on other product identifiers. Use Product Data Fields to specify which fields should support partial‑text search. For example, to make a product field or a custom field searchable using partial text, enter modelNumber as the Product Data Field. If the identifier is a product attribute, such as vendorNumber, enter vendorNumber in the Product Data Fields list.
After you performing a rebuild, you must update the newly added data fields in Marketing > Commerce Search v3 > Attribute Controls with the following settings for all applicable websites and languages, followed by another rebuild to apply the changes:
- Dynamic Facetable – False
- Exact Match – True
- Searchable – True
After you enable N‑grams and configure the appropriate fields, reindex the catalog a couple of times so the updated N-gram tokenization applies to all relevant fields.
Verify canonical filter
Test how the Canonical filter setting influences query expansion behavior.
Enable Search Query Expansion and Canonical filter in the Admin Console. Perform searches using long-tail queries that typically return few or no results.
Compare results with the Canonical filter setting turned on and off:
- Verify that query expansion triggers when the original query returns few results.
- Verify that applied filters (such as category, brand, or attribute filters) do not incorrectly prevent query expansion when Canonical filter is enabled.
- Verify that query expansion evaluates results against a broader catalog view rather than only the filtered result set.
Apply filters, sorting, and pagination, and confirm the following:
- Filtering does not prevent relevant expanded results from appearing when appropriate.
- Sorting does not negatively affect the relevance of expanded results.
- Pagination returns consistent and expected results across pages.
Verify that the final search results remain unchanged in structure:
- Confirm that Canonical filter does not directly modify the products returned.
- Confirm that the setting only affects when query expansion is triggered.
Browse categories and confirm that:
- Category navigation behavior is not affected by the Canonical filter setting.