Search pipeline

17 min

📘 Note By default, Configured Commerce uses Commerce Search v2 (Elasticsearch version 7.10) as the search provider for customer storefronts. The following search pipelines only apply to Commerce Search v1 (Elasticsearch version 5.5). If you upgrade to Commerce Search v2 (Elasticsearch version 7.10) or Commerce Search v3 (Elasticsearch version 7.10 and Google Cloud Retail Search/Vertex AI Search for Commerce), see Third party search pipeline.

CategoryIndexerPipeline

CreateElasticsearchCategory

Order	Pipe	Description
100	CreateElasticsearchCategory	Create a new ElasticsearchCategory object using the parameter object's fields and assigns it to result.ElasticsearchCategory.

GetIndexableCategories

Order	Pipe	Description
100	GetIndexableCategoriesQuery	Creates an Entity Framework Category query using the parameter object's fields and assigns it to result.IndexableCategories.

MapElasticSearchCategoryProperty

Order	Pipe	Description
100	MapElasticSearchCategoryCustomProperties	If the parameter.SearchBoost.IsQueryable is true and parameter.PropertyDefinition.IsCustomProperty is true sets result.Property to a new Nest TextProperty.

CategorySearchPipeline

RunCategorySearch

Order	Pipe	Description
100	InitializeSearchProperties	Retrieves the current SiteContext, ElasticsearchIndex and ElasticsearchQueryBuilder and assigns them to the result object.
200	GetSettings	Retrieves SearchGeneralSettings and FuzzySearchSettings and assigns them to the result object.
300	GetQueryFields	Assigns the result.CategorySearchFields object with the fields that are queryable.
400	FormPaging	Assigns result.StartRow and result.PageSize from the parameter object.
500	FormCategoryExcludeList	Finds the disallowed categories for the current user (using the FormProductFilter pipeline) and caches this value. The value is then assigned to result.DisallowedCategories.
600	FormCategoryExcludeFilter	If there are any disallowedCategories adds a new negated Terms query to result.AllFilters using result.DisallowedCategories on the categoryId field.
700	FormLanguageFilter	Adds a new MatchQuery using the result.SiteContext.LanguageDto.LanguageCode on the languageCode field to result.AllFilters.
800	FormWebsiteFilter	Adds a new MatchQuery using the result.SiteContext.WebsiteDto.Id on the websiteId field to result.AllFilters.
900	CombineFilters	Creates a BooleanQuery (AND) of all filters in result.AllFilters and assigns it to result.CombinedFilter.
1000	FormMainSearchQuery	Creates a MultiMatchQuery of type CrossFields using parameter.Criteria and result.CategorySearchFields and apply boosts. Creates a MultimatchQuery of type PhrasePrefix using parameter.Criteria and result.CategorySearchFields and apply boosts. Adds both of these queries to result.AllQueries.
1100	FormFuzzySearchQuerry	If fuzzy search is enabled, creates a MultiMatchQuery with fuzziness set to FuzzySearchSettings.MaxEdits using parameter.Criteria and result.CategorySearchFields and apply boosts. Adds this query to result.AllQueries.
1200	CombineQueries	Creates a BooleanQuery (OR) of all queries in result.AllQueries and assigns it to result.CombinedQuery.
1300	ApplyQueryAndFilter	Creates a SearchDescriptor with a FunctionScore query using result.CombinedQuery and result.CombinedFilter, with a FieldValueFactor function using the boost field and assign it to result.SearchDescriptor.
1400	ApplyPaging	Sets result.SearchDescriptor's From and Size variables using result.StartRow and result.PageSize.
1500	ApplySortOrder	Sets result.SearchDescriptor's sort to sort on the sortOrder field in descending order then on _score in descending order.
1600	PerformSearch	Uses result.ElasticsearchIndex.Client to call Elasticsearch using result.SearchDescriptor and assigns the result to result.SearchResponse.
1700	ProcessSearchResponse	If result.SearchResponse is not null and is valid, creates a new CategorySearchResult with the search result count and an IEnumerable of CategorySearchResultDto objects using result.SearchResponse.Documents then assigns this to result.CategorySearchResult.

ContentIndexerPipeline

CreateElasticsearchContent

Order	Pipe	Description
100	CreateElasticsearchContent	Creates a new ElasticsearchContent using the parameter object's fields and assigns it to result.ElasticsearchContent. Fields that vary based on language, persona are populated by filtering parameter.IndexableContentItem.Fields for the field with the appropriate context.

GetIndexableContentItems

Order	Pipe	Description
100	GetIndexableContentItemQuery	Creates an Entity Framework IndexableContentItem query using the parameter object's fields and assigns it to result.IndexableContentItems.

ContentSearchPipeline

RunContentSearch

Order	Pipe	Description
100	ValidatePageSize	Sets result.PageSize based on parameter.PageSize, if it is less than or equal to zero set to 10, if it is greater than 1000 set to 1000 or else set to parameter.PageSize.
200	FormQuery	Creates a BooleanQuery (OR) of two MultiMatchQuery queries for the content fields (title, body, metaDescription, and metaKeywords) one of type CrossFields and one of type PhrasePrefix and assigns it to result.Query.
300	FormFilter	Creates a BooleanQuery (AND) of TermQueries to filter on website, language, deviceType, persona and not restricted pages and assigns it to result.Filter.
400	PerformSearch	Creates a new SearchRequest using result.Query, result.Filter and result.PageSize, then performs the search and assigns the response to result.SearchResponse.
500	ProcessSearchResponse	Creates an IEnumerable of ContentSearchResultsDto from the result.SearchResponse.Document fields (title, pageType, and URL) and assigns it to result.ContentSearchResults.

ProductIndexerPipeline

CreateElasticsearchProduct

Order	Pipe	Description
100	CreateElasticsearchProduct	Create a new ElasticsearchProduct object using the parameter object's fields and assigns it to result.ElasticsearchProduct.
200	SetSpellingCorrection	Joins ElasticseachProduct fields (ShortDescription, Content, ManufacturuerItem, Name, Specifications, ErpNumber, CategoryNames, and FilterNames) into a string and assigns it to result.ElasticsearchProduct.SpellingCorrection.

GetIndexableProducts

Order	Pipe	Description
100	GetIndexableProductsSqlStatementParts	Creates a string with the Optimizely default SQL query to get the correct products and fields from the database and assigns it to result.SqlStatement.Assigns result.IncremenalFilter with the proper SQL snippet if parameter.IsIncremental is true.
200	CombineIndexableProductsSqlStatementParts	Combines result.SqlStatement, result.CustomFields, and result.IncrementalFilter and assigns it to result.FormattedSqlStatement.
300	PerformIndexableProductsSqlQuery	Gets an IEnumerable using the SQL query from result.FormattedSqlStatement and assigns it to result.IndexableProducts.

MapElasticSearchProductProperty

Order	Pipe	Description
100	MapElasticSearchProductCustomProperties	If the parameter.SearchBoost.IsQueryable is true and parameter.PropertyDefinition.IsCustomProperty is true sets result.Property to a new Nest TextProperty.

ProductSearchPipeline

FormProductFilter

Order	Pipe	Description
100	GetSettings	Retrieves SearchGeneralSettings and ProductRestrictionsSettings and assigns them to the result object.
200	FormWebsiteFilter	Adds a new MatchQuery using the result.SiteContext.WebsiteDto.Id on the websiteId field to result.AllFilters and assigns it to result.WebsiteFilter.
300	FormLanguageFilter	Adds a new MatchQuery using the result.SiteContext.LanguageDto.LanguageCode on the languageCode field to result.AllFilters and assigns it to result.LanguageFilter.
400	FormCategoryFilter	Creates a new PrefixQuery to search a single category or the whole category tree based on parameter.ProductSearchParameter fields (SearchCriteria and IncludeSubcategories) then adds it to result.AllFilters and assigns it to result.CategoryFilter.
500	FormRestrictionGroupFilter	If Restriction Groups are enabled, uses the ProductSearchPipeline's FormRestrictionGroupFilter pipe to generate a new filter then adds the filter to result.AllFilters and assigns it to result.RestrictionGroupFilter.
600	FormAttributeValueFilter	Creates PrefixQueries for attributes based on parameter.ProductSearchParameter.AttributeValueIds and combines them into a BooleanQuery. That BooleanQuery is added to result.AllFilters and assigned to result.AttributeValueFilter.
700	FormAllowedProductsFilter	Creates MatchQuery objects for each product in parameter.ProductSearchParameter.AllowedProductIds and combines them into a BooleanQuery. That BooleanQuery is add to result.AllFilters and assigned to result.AllowedProductsFilter.
800	FormPriceRangeFilter	If parameter.ProductSearchParameter.PriceFilters has any values a new MatchQuery is created for each filter and combined into a BooleanQuery (OR) then assigned to result.PriceRangeFilter and added to result.AllFilters. If parameter.ProductSearchParameter.PriceFilters has no values and parameter.ProductSearchParameter.MinimumPrice or parameter.ProductSearchParameter.MaximumPrice have been supplied a new Range query is created using the supplied min/max values with defaults of 0 and 1000000 if either was omitted. That Range query is then added to result.AllFilters and assigned to result.PriceRangeFilter. If no filters were supplied and neither the minimumPrice or maximumPrice were supplied nothing is done and the pipe returns.
900	CombineFilters	Creates a BoolQuery (AND) of all filters in result.AllFilters and assigns it to result.CombinedFilter.

FormRestrictionGroupFilter

Order	Pipe	Description
100	GetRestrictionGroupFilterFromCache	Attempts to retrieve the restriction group filter from cache based on the current BillTo/ShipTo and assigns it to result.RestrictionGroupFilter. If there was a cached filter the pipeline exits. if there was no cached filter the pipeline continues to the next pipe.
200	GetDefaultVisibilityFilters	Creates a new MatchQuery on the restrictionGroups field to match products with no restriction groups and adds it to result.DefaultVisibilityFilters. Creates a Terms query on the defaultVisibility field with values of empty and show then adds it to result.DefaultVisibilityFilters.
300	GetWebsiteRestrictionGroupQuery	Creates an EF query to get the active restriction groups based on the current website and assigns it to result.WebsiteRestrictionGroupQuery.
400	GetWebsiteRestrictionGroups	Uses result.WebsiteRestrictionGroupQuery to get active restriction groups then assigns them to result.WebsiteRestrictionGroups. If there are any active restriction groups a BoolQuery (AND) is created from result.DefaultVisibilityFilters and assigned to result.RestrictionGroupFilter then the pipeline exits. If there are no active restriction groups the pipeline continues to the next pipe.
500	GetCustomerRestrictionGroupIds	If there is no BillTo the pipeline does nothing and continues to the next pipe. If BillTo or ShipTo are populated result.CustomerRestrictionGroupIds is set to the applicable restriction group IDs from result.WebsiteRestrictionGroupQuery.
600	GetShowOnlyRestrictionGroupIds	Adds all restriction group IDs from result.WebsiteRestrictionGroups where the restrictiongroup type is "ShowOnly" and is an applicable restriction group (result.CustomerRestrictionGroupIds) to result.ShowOnlyRestrictionGroupIds.
700	GetShowRestrictionGroupIds	Adds all restriction group IDs from result.WebsiteRestrictionGroups where the restrictiongroup type is "Show" and is an applicable restriction group (result.CustomerRestrictionGroupIds) to result.ShowRestrictionGroupIds.
800	GetHideRestrictionGroupIds	Adds all restriction group IDs from result.WebsiteRestrictionGroups where the restrictiongroup type is "Hide" and is an applicable restriction group (result.CustomerRestrictionGroupIds) to result.HideRestrictionGroupIds.
900	FormShowOnlySubFilter	Creates a Terms query on the field restrictionGroups with the values from result.ShowOnlyRestrictionGroupIds then adds it to result.SubFilters and assigns it to result.ShowOnlySubFilter.
1000	FormShowSubFilter	If there are no filters in result.DefaultVisibilityFilters and no filters in result.ShowRestrictionGroupIds continue to next pipe. Creates BoolQuery (AND) from the filters in result.DefaultVisibilityFilters.Creates MatchQuery objects on the field restrictionGroups for all restriction groups in result.ShowRestrictionGroupIds. Combines the previously created queries into a BoolQuery (OR) then adds it to result.SubFilters and assigns it to result.ShowSubFilter.
1100	FormHideSubFilter	If there are no filters in result.HideRestrictionGroupIds continue to next pipe. Creates MatchQuery (NOT) object on the field restrictionGroups for all restrcition groups in result.HideRestrictionGroupIds. Combines the previously created queries into a BoolQuery (AND) then adds it to result.SubFilters and assigns it to result.HideSubFilter.
1200	CombineFilters	If result.ShowOnlySubFilter is empty and result.SubFilters is empty continue to next pipe. Create a BoolQuery (AND) fromt he filters in result.SubFilters. Combine BoolQuery just created and result.ShowOnlySubFilter into a new BoolQuery (OR) and assign it to result.RestrictionGroupFilter.
1300	AddRestrictionGroupFilterToCache	Adds result.RestrictionGroupFilter to the cache based on the current BillTo/ShipTo.

RunProductSearch

Order	Pipe	Description
100	GetSettings	Retrieves SearchGeneralSettings, CatalogGeneralSettings, FuzzySearchSettings, SponsoredSearchSettings and SearchSuggestionsSettings then assign them to the result object. Set result.UsingListPrice, result.CanSortPrice, and result.PriceFacetingEnabled from various settings objects.
200	InitializeSearchProperties	Retrieve and assign values for result.BoostHelper, result.ElasticsearchIndex, result.ElasticsearchQueryBuilder, result.PhraseSuggestConfiguration, and result.ProductSearchFacetProcessor.
300	FormSortOrder	Populate result.SortOrderFields based on parameter.ProductSearchParameter fields.
400	ApplySortOrder	Create a new Nest SortDescriptor<ElasticsearchProduct> and set the sort based on result.SortOrderFields.
500	GetQueryFields	Adds the fields that are queryable using data from the search.SearchBoosts table to result.ExactMatchFields, result.PrefixMatchFields, and result.PhraseMatchFields. Adds the fields that are part of fuzzy match (categoryNames and shortDescription) to result.FuzzyMatchFields.
600	FormSearchCriteriaQuery	If parameter.ProductSearchParameter.SearchCriteria is blank continue on to next pipe without doing anything else. Creates and assigns queries for result.ExactMatchQuery, result.PrefixMatchQuery, result.PhraseMatchQuery and result.FuzzyMatchQuery (if enabled) based on the fields assigned in GetQueryFields pipe. All new queries added to result.AllQueries.
700	FormCustomerNameQuery	If parameter.ProductSearchParameter.SearchCriteria is blank for BillTo and ShipTo are null continue on to next pipe without doing anything else. If BillTo is not null create a PrefixQuery on the customerNames field and match on BillTo.Id + searchCriteria. If ShipTo is not null create a PrefixQuery on the customerNames field and match on ShipTo.Id + searchCriteria. Create a BoolQuery (OR) from the previously created queries then add it to result.AllQueries and assign it to result.CustomerNameQuery.
800	FormSearchWithinQuery	If parameter.ProductSearchParameter.SearchWithin is blank continue on to the next pipe without doing anything else. Create a MultiMatchQuery of type CrossFields for Product on the result.ExactMatch fields querying on parameter.ProductSearchParameter.SearchWithin and assign it to result.SearchWithinQuery.
900	CombineQueries	Create a BoolQuery (OR) from result.AllQueries. If result.SearchWithinQuery is not null create a BoolQuery (AND) with it and the previously created BoolQuery. Assign the resulting BoolQuery to result.CombinedQuery (either just the first BoolQuery or the combined BoolQuery).
1000	FormProductFilter	Use the ProductSearchPipeline's FormProductFilter pipeline to get the product filter and assign it to result.FormProductFilterResult.
1100	FormCategoryAggregation	If parameter.ProductSearchParameter.DoFacetedSearches is false continue on to the next pipe without doing anything else. If parameter.ProductSearchParameter.SearchCriteria is blank and parameter.ProductSearchParameter.IncludeSubcategories is false continue on to the next pipe without doing anything else. Using result.FormProductFilterResult.AllFilters create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the categories field and add it to result.AllAggregations and assign it to result.CategoryAggregation.
1200	FormAttributeValueAggregation	If parameter.ProductSearchParameter.DoFacetedSearches is false or result.SearchGeneralSettings.EnableAttributeFilters is false continue on to the next pipe without doing anything else. If result.SearchGeneralSettings.FilterOnSearch if false and parameter.ProductSearchParameter.SearchCriteria is not blank continue on to the next pipe without doing anything else. Using result.FormProductFilterResult.AllFilters create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the filters field and add it to result.AllAggregations and assign it to result.AttributeValueAggregation. If parameter.ProductSearchParameter.GetAllAttributeFacets is false continue on to the next pipe without doing anything else. Using result.FormProductFilterResult.AllFilters where the filters are not equal to result.FormProductFilterResult.AttribueValueFilter create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the filters field and add it to result.AllAggregations and assign it to result.AllAttributeValueAggregation.
1300	FormPriceRangeAggregation	If parameter.ProductSearchParameter.DoFacetedSearches is false or result.PriceFacetingEnabled is false continue on to the next pipe without doing anything else. Using result.FormProductFilterResult.AllFilters where the filters are not equal to result.FormProductFilterResult.PriceRangeFilter create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the priceFacet field and add it to result.AllAggregations and assign it to result.PriceRangeAggregation.
1350	FormPersonaScoreFunctionList	If parameter.SiteContext.PersonaDtos contains personas with EnableSearchBoost true, the result ScoreFunctions collection will be populated with one WeightFunction for each persona with a TermQuery on the Persona.Id and boost value as a Weight. The ScoreFunctions collection will be added to the query by the ApplyQueryAndFilter pipe.
1400	CombineAggregations	If parameter.ProductSearchParameter.DoFacetedSearches is false continue on to the next pipe without doing anything else. Call result.CombinedAggregation.Filter for each aggregation in result.AllAggregations and assign the result to result.CombinedAggregation.
1500	FormDidYouMeanSuggestion	If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else. Call result.PhraseSuggestConfiguration.Configure with productSearchParameter.SearchCriteria and result.SearchSuggestions.DidYouMeanThreshold (limited to 0 to 5) and assign the value to result.DidYouMeanSuggestion and add it to result.AllSuggestions.
1600	FormAutoCorrectSuggestion	If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else. Call result.PhraseSuggestConfiguration.Configure with productSearchParameter.SearchCriteria and result.SearchSuggestions.AutoCorrectThreshold (limited to 0 to 5) and assign the value to result.DidYouMeanSuggestion and add it to result.AllSuggestions.
1700	CombineSuggestions	If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else. Call result.CombinedSuggestion.Phrase for each suggestion in result.AllSuggestions and assign the result to result.CombinedSuggestion.
1800	ApplySuggestion	If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else. Call result.SearchDescriptor.Suggest with result.CombinedSuggestion and assign the result to result.SearchDescriptor.
1900	PerformAggregationSearch	If parameter.ProductSearchParameter.DoFacetedSearches is false or result.AllAggregations is empty continue on to the next pipe without doing anything else. Create an AggregationSearchDescriptor from result.CombinedQuery and result.FormProductFilterResult.CombinedFilter then use that to query Elasticsearch. Assign result from Elasticsearch to result.AggregationSearchResponse.
2000	ProcessCategoryAggregation	Use result.ProductSearchFacetProcessor.ProcessCategoryFacets to process the facets and assign the return value to result.ProductSearchResult.CategoryDtos.
2100	ProcessAttributeValueAggregation	Use ProductSearchFacetProcessor.ProcessAttributeTypeFacets to process the facets and assign the return value to result.ProductSearchResult.AttributeTypeDtos.
2200	ProcessPriceRangeAggregation	Use ProductSearchFacetProcessor.ProcessPriceRangeFacets to process the facets and assign the return value to result.ProductSearchResult.PriceRangeDto.
2300	PerformSponsoredSearch	If result.SponsoredSearchSettings.Enabled is false or parameter.IsAutocomplete is true or this is a filtered search or if this is not sorted by relevance or parameter.ProductSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else. Create a new TermQuery on the isSponsored field. That is added to a new query along with result.CombinedQuery and result.FormProductFilterResult.CombinedFilter with a max result length of result.SponsoredSearchSettings.Limit or 5. This is used to query Elasticsearch. The results are assigned to result.SponsoredSearchResponse.
2400	ProcessSponsoredSearchResponse	If result.SponsoredSearchResponse is null continue on to the next pipe without doing anything else. Create a new QueryContainerDescriptor<ElasticsearchProduct> to exclude all sponsored result Ids and assign it to result.ExcludeSponsoredProductsFilter. Set result.ProductSearchResult.Count to the sponsored search's total results. If this is the first page of results (result.StartRow equals 0), add the returned products to result.ProductSearchResult.Products and set result.PageSize to result.PageSize minus the number of sponsored results returned. If this is not the first page of results (result.StartRow does not equal 0), set result.StartRow to result.StartRow minus the number of sponsored results returned.
2500	ApplyQueryAndFilter	If result.ExcludeSponsoredProductsFilter is not null make a new new boolean query with result.FormProductFilterResult.CombinedFilter and result.ExcludeSponsoredProductsFilter. If result.CombinedQuery is null create a SearchDescriptor using only the above created filter or result.FormProductFilterResult.CombinedFilter. If result.CombinedQuery is not null and result.SearchGeneralSettings.EnableProductBoost is true, create a SearchDescriptor with a FunctionScore query using result.CombinedQuery and the above created filter or result.FormProductFilterResult.CombinedFilter with a FieldValueFactor function using the boost field and assign it to result.SearchDescriptor. If result.ScoreFunctions collection is not null or empty, these functions will be added to the FunctionScore query. This will provide boosting of products with certain personas. If result.CombinedQuery is not null and result.SearchGeneralSettings.EnableProductBoost is false, createa SearchDescriptor using result.CombinedQuery and the above created filter or result.FormProductFilterResult.CombinedFilter.
2600	ApplyPaging	Set from and size on result.SearchDescriptor using result.StartRow and result.PageSize.
2700	PerformSearch	Set from and size on result.SearchDescriptor using result.StartRow and result.PageSize.
2800	ProcessSearchResponse	If result.SearchResponse is null or is not valid return a SearchFailure code with relevant message. Add returned products from SearchResponse to result.ProductSearchResult.Products. Add the total count of returned products to result.ProductSearchResult.Count. If result.CanSortByPrice is false or result.UsingListPrice is false and result.ProductSearchResult.Count is greater than result.CatalogGeneralSettings.PriceSortMaximum, then set result.CanSortByPrice to false, set result.ProductSearchResult.SortOrder to result.SortBy and set result.ProductSearchResult.SortOptions to parameter.SortOptions where PriceLowToHigh and PriceHighToLow are removed. If result.CanSortByPrice is true or result.UsingListPrice is true and result.ProductSearchResult.Count is less than or equal to result.CatalogGeneralSettings.PriceSortMaximum, then set result.ProductSearchResult.SortOptions to parameter.SortOptions.
2900	ProcessDidYouMeanSuggestion	If result.SearchResponse.Suggest is null or result.SearchResponse.Suggest does not contain the key "didyoumean" or didYouMeanSuggestions.Length is not equal to 1 continue on to the next pipe without doing anything else. Populate result.ProductSearchResult.DidYouMeanSuggestions from the result.SearchResponse.Suggest entry for "didyoumean".
3000	ProcessAutoCorrectSuggestion	If result.SearchResponse.Suggest is null or result.SearchResponse.Suggest does not contain the key "correction" or autoCorrectSuggestions.Length is not equal to 1 continue on to the next pipe without doing anything else. Populate result.ProductSearchResult.AutoCorrectSuggestion from the result.SearchResponse.Suggest entry for "correction".