Optimizely Configured Commerce SDK v5.1 Release
25 min
Optimizely is excited to announce our v5.1 release, which continues to reflect a focus on delivering a scalable and extensible solution that meets the changing business needs of B2B manufacturers and distributors. The 5.1 release focuses on improving the experience of the business user to manage their digital platform and enhance the B2B end-user experience.
The v5.1 SDK download is now available. There were a number of changes that could impact customers upgrading to 5.1. Please review this upgrade from 4.6 to 5.1 article for instructions and considerations before upgrading.
This release includes the following new features and functionality:
Spire CMS
Spire is both a new CMS and a new, from the ground up, reference site. It uses different technology than our current Classic CMS, but the main ideas are the same - pages are constructed from CMS widgets, widgets are displayed on the pages and the CMS/Javascript Framework use APIs to retrieve data.
The primary technical goals for the rewrite of the Optimizely Configured Commerce front-end and CMS are:
- Improved upgradability for the frontend
- More composable commerce pages
- Simple CMS Data Structure
- Improved page editing experience
- SEO/Serverside rendering
- Lighter Javascript payloads
While not a technical goal, another intent of this initiative is to empower business users to have more control over the site without developer intervention and to keep implementation costs competitive with other platforms.
Search Improvements
- Exposed search score in the CMS.
- Search Abstraction – Rearchitected search APIs for use with any search provider.
- Hawksearch Connector – Customers can now use Hawksearch as a search provider out of the box Hawksearch documentation.
- Updated Search Abstraction methods to support Search Rebuild version 2 (see Third party search pipelines).
- Disabled Admin Console settings for Elasticsearch when using a third-party search provider.
- Updated Search indexes to include "Is Stocked" flag on warehouse level Adjusted the search index to only index custom properties that are marked searchable/queryable.
- Pipelined the search process to allow for custom analyzers (See ConfigureIndexSettings).
- Search improvements related to streamlining Products API.
- Obsoleted ICatalogPipeline.GetAvailability with message "Use PopulateAvailabilities pipeline instead" and replaced our usages of it with ICatalogPipeline.PopulateAvailabilities. See the PopulateAvailabilities article for more information.
- Obsoleted ICatalogPipeline.CanConfigure with message “Use IProductUtilities.CanConfigure instead” and replaced our usages of it with IProductUtilities.CanConfigure. See the Overview of Pipelines article for more information on specific pipelines.
- Updated how Configured Commerce converts bidirectional synonyms to save them as a Word-Synonym pair instead of comma-separated in the Synonym column.
- Added functionality to clear the Category cache when rebuilding the site index (See Understanding Indexing).
- Added the ability to search for a variant by product number, even if another product has a similar or the same manufacturer number.
- Added the ability to override default Elasticsearch document objects used for mapping properties to support custom fields in product indexing.
- Added mechanism to capture Autocomplete Searches to the data layer.
- Added indexing and querying of new attribute additions to ElasticSearch.
- Improved search rebuild by capturing better information when search indexing fails with HttpStatusCode.
- Removed IProductSearchParameter and IProductSearchResult interface interfaces and now use ProductSearchParameter and ProductSearchResult classes directly - We added some new functionality to the search process overall, which required a breaking change to the interface IProductSearchResult and IProduceSearchParameter interfaces. When doing so, we determined that the specific method interfaces did not conform to our standard, upgradable pattern of having a parameter collection and results collection. Although this is a major breaking change to the site, it was a good time to do it and will help moving forward to prevent additional breaking changes. See this article for the impact and steps to resolve - the site may not compile or may require some refactoring in order for search to work properly if you have any customizations in the affected areas.
- Added ResultBase as a base class to ProductSearchResult.
- Removed IProductSearchParameter interface and use ProductSearchParameter directly.
- Added ParameterBase as a base class to ProductSearchParameter. Moved ProductSearchParameter.Properties dictionary to base class (ParameterBase).
- Reworked Elasticsearch indexer to be backwards compatible.optimizely-configured-commerce-sdk-v5-1-release.mdxoptimizely-configured-commerce-sdk-v5-1-release.mdx
Value Added Tax (VAT)
In the European Union, VAT is charged by all VAT-registered companies to sellers within the same country. Sales to other countries do not require VAT. Enable and configure VAT on your Configured Commerce website to meet this tax requirement.
The general process to use VAT on your Configured Commerce site is to enter your VAT codes and VAT rates, then turn the Enable VAT setting on to show VAT fields in the Admin Console and VAT calculations on your site. Configure two other tax settings, including selecting the VAT tax calculator and shipping VAT code, after you enable VAT.
ReCAPTCHA
- Added a tooltip to the Enable reCAPTCHA setting in the Admin Console (see Enable reCAPTCHA)
- Added functionality to require reCAPTCHA on login when users request to generate any emails in Classicoptimizely-configured-commerce-sdk-v5-1-release.mdxoptimizely-configured-commerce-sdk-v5-1-release.mdx
Performance Improvements
- Added a non-unique index to the CustomerProduct table to improve search index rebuild performance
- Improved performance of Category-Product importing
- Improved performance of GetContentItem fields in Classic
- Added a search rebuild version 2 process to improve speed and performance of index builds. Optimizely encourages customers to upgrade to version 2. However, some customizations may require additional effort to ensure proper functionality. Customers should verify upgradability with their partner. Customers also will need to opt in on version 2, although the new default for resets to default search will be version 2. Note that the version 2 option is only available to ISC_Implementer roles.
- Removed the record limit for the OData export process to increase performance of data exports (see Admin API architecture)
- Improved Category API performance
- Improved Cart page performance for carts with large item counts in Classic
- Improved the performance of the query used to build the search index
- Created a script to index the product Model Number in SQL to improve site performance
- Improved performance of variants by reducing individual SQL calls
- Improved Search Query for speed
- Refactored mechanism for top sellers pool to improve performance
- Added code to add or alter indexes for performance
- Improved performance of image and other product data exports
- Improved how to handle changes to categories and the impact on partial indexing
- Made updates to improve startup performance
Admin Console
- Updated icons in the Admin Console
- Enhanced the Google API key site configurations
- Added the ability to translate Configuration Sections and corresponding options in the Admin Console
- Modified logging process for containerized sites so that logging flows through Elasticsearch logs (see Review application logs)
- Removed authentication credentials for ERP Connectors in debug logs
- Added a setting to the Admin Console to make $0 pricing valid for configured products (see Working with pricing)
- Added a configurable site message for products added to the cart when the product has invalid pricing data (see Working with pricing)
- Added ability to display different images based on site and language. See Display translated images based on language.
- Removed the ability to create a website in Master Edit mode for ISC_Implementer users
- Added a setting to the Admin Console to support multiple email domains, so a global default and website-specific email domain hosts
- Added the ability to clear the memory cache using the Clear Cache option in the Debug menu of the Admin Console (See Debugging Tools)
- Added the Use Updated Rounding Logic setting to the Admin Console
- Added guard rails around the websites table so that critical entries aren't removed by accident
- Updated the styling for the Admin Console and Spire CMS to match Optimizely branding
- Implemented a sortable table pattern in the Order Approvals table
- Created a mechanism to clear the static http client provider to the existing "Clear Cache" function in the Admin Console
- Update tooltip text for timeout setting to include anything beyond 20 min is a "violation" of the PCI compliance rules
- Added the ability to change website names in Kubernetes environments for users with the ISC_System role
- Added step to decode HTML Entities during product import for Short Description field (see Creating a Product and Importing Products Using the Import Template)
- Added a debug message that triggers when the timeout is reached for real-time pricing calls
- Added a CMS Type setting to the Admin Console to select if a site has Classic or Spire
- Added the Show Inventory on Cart and Real Time Inventory Batch Size settings to control if to call for inventory from the cart and the paging of items sent in the real time call
- Updated Budget_CreateBudgetForLevel site message to No budget configured. Please configure a budget to continue.
- Updated translations to automatically handle `replace("")` process
- Updated the Admin Console to stay active during image uploads so it won't timeout after 15 minutes
- Implemented a centralized logging solution to remove logs from the database
- Added functionality to display Integration Job Logs from centralized logging in the Admin Console
- Added functionality to display the application logs directly from the external log resource in the Admin Console, if available
- Added functionality to send Elmah logs to external logging
- Added functionality to send integration job logs to external logging
- Added the Display Logs from Database setting to the Admin Console to enable showing the log location (for display), but setting shows read-only if appsetting found in file Added the ability to allow ErpOrderNumber within the Shipment Confirmation email template
- Added the View only product option to the Product Restrictions tab as a way to mark a product viewable, but not purchasable
- Modified the translation processing so it won't re-read the dictionary if something is missing in cache
- Added an option to create secure cookies if a site is accessed via HTTPS
- Added a Product Search Path setting and option in the Translation Dictionary to translate the search URL
- Expanded the Document.Name field to 255 characters
- Updated export functionality to improve performance with a large number of products and eliminate errors/failures
- Added "Is Stocked" flag to the Warehouse level for individual products
- Added a Real Time Pricing Batch Size setting to control the paging of the items sent in the real time calloptimizely-configured-commerce-sdk-v5-1-release.mdxoptimizely-configured-commerce-sdk-v5-1-release.mdx
Storefront
- Improved error messaging when a customer saves an order then revisits it and all products are now restricted
- Modified sorting of Order History page to return orders by descending date and order number. See Order and invoice history.
- Modified sorting of Invoice History page to return invoices by descending date and invoice number. See Order and invoice history.
- Modified the Remove All Items From Cart behavior to delete each cart line versus deleting the CustomerOrder object
- Added RMA Product List buttons to the RMA page
- Added the RMA option to Order Details page
- Updated filtering so attribute filters with hidden child products don't display as options
- Created an alert for adding saved orders to carts that asks whether to add to the current cart or change the ship-to to match the saved order
- Added the ability to filter and analyze dimensions/measurements in the storefront
- Configured Forgot Password to not leak information
- Improved how opening a translated catalog/product/brand page URL changes a site's language in Classic and Spire
- Added a Lists import template to the Lists functionality in the Admin Console (See Share Lists with Multiple Customers)
- Updated promotions to automatically include any child variants when they are applied to a category
- Changed display of facet to show the name instead of a blank label in the storefront
- Improved real-time pricing experience on Lists
- Added the ability to use storefront Saved Credit Cards from Paymetric
SSO
- Modified the SSO redirect URLs to always be HTTPS
- Added settings to enable the ability to use OpenId Connect as an SSO provider (See Set Up OpenId Connect SSO)
- Added new Admin Console settings for Windows SSO: "Storefront Caption", "Storefront Application ID URI", "Admin Console Caption", and "Admin Console Application ID URI" (see Set up Microsoft Azure SSO)
- Added functionality to honor role claim mapping when a user signs in using OpenId Connect SSO (see Set up OpenId Connect SSO)
- Added a system setting to disable automatically adding the ISC_User role to users auto-created from SSO in the Admin Console
- Added the ability to use OpenId Connect with Azure ADB2C
Classic CMS
- Added hreflang flag to the sitemap for sites configured for multiple locales in Classic (see Overview of multi-language and translations)
- Replaced FlexSlider.js with Embla.js, an open source carousel (calls should be the same as before, but autoplay is not available as an option)
- Improved CKFinder to use native S3 access to improve performance when loading many files in one directory
- Added ability to switch accounts to Storefront headers for Responsive and Distribution system themes
- Updated Chrome and Chromedriver to ensure cookies work correctly with future CMS versions
- Updated CKFinder to the current version
- Added the ability to export My Lists to a CSV file in Classic
- Updated how to handle internal URL redirects using a relative path in Classic
- Added option under the Classic CMS gear icon to Enable Search Data Mode to search in the CMS shell
- Added a Search Button to CkFinder (Media Library) to help filter results and limited folders to show up to 1000 files at a time in folders
- Created unique, language-specific URLs for home pages in Classic
Integrations
- Payment Gateways
- Made Stripe payment gateway settings website-specific (see Connectors for Stripe)
- Updated the Stripe payment gateway to allow sending of Live Secret Key and Test Secret Key as parameters during a Transaction Request
- Updated the Payeezy payment gateway to send Level 2 / Level 3 information in on Authorization and Sale (00 and 01) transactions (See Connectors for Payeezy)
- Updated the payment service to allow designation of SOAP vs REST API for Payeezy
- Updated the Payeezy payment gateway to truncate postal codes to 7 characters instead of 6
- Added the tax for Level2/Level3 as a part of the Level2.Tax1Amount for the Payeezy SOAP API
- Added the ability to maintain website-specific configurations for the VantivExpress payment gateway
- Added ability to add prefix to AccountNumber in VantivExpress (see Connectors for VantivExpress)
- Added the ability to designate payment gateway settings as a call parameter for VantivExpress
- Added Payment Gateway Vault functionality to the Orbital payment gateway (See Connectors for Orbital)
- Added PayTrace as a Cloud payment gateway (see Connectors for PayTrace for more details)
- Added Evo payment gateway integration. See Connectors for Evo.
- Updated the CenPOS payment gateway to accept the success code for a corporate card
- Added ability to designate payment gateway Merchant ID as call parameter for CenPOS
- Added a connector for the Bambora payment gateway (payment processing only) to the Admin Console
- Added Payment Profiles to the Bambora payment gateway connector. See Connectors for Bambora.
- Removed mask-number flag from Paymetric Iframe to avoid issues with IE
- Updated the Paymetric iframe settings to display for the ISC_Implementer role
- Added the ability to render a different payment gateway iframe (Paymetric) through settings
- Added toggle to send Paymetric Expiry as MM/YY
- Updated handling of multiple Paymetric Merchant ID's to inject credentials, if needed
- Updated the Authorize.Net payment gateway to allow sending of Merchant ID and Transaction Key as parameters during a transaction request (see Overriding settings in custom code)
- Added a setting to concatenate the Email and CustomerNumber for saved payment profiles for Authorize.Net
- Added new settings for client IP and successful response code to the Authorize.Net credit card logic updates
- Implemented Elavon as a payment gateway connector
- Updated the Moneris payment gateway to allow Store ID and API token to be sent as parameters and allow overrides of the MerchantId
- Added EbizCharge as a new payment gateway
- Added APS Payments as a new payment gateway connector option (see Connectors for APS Payments)
- Updated the workflow of submitting tokens to APS Payments
- Added the Adyen payment gateway
- ERPs
- Updated the ARCustomerMnt call from SX.e v11 to look in both the cErrorMessage field and the returnData field for the customer number
- Updated the unit price rounding option in the SX.e price calculator
- Added order-level notes to the Acumatica Order Submit integration job
- Added the ability to set the MarkForPO field to null in the Acumatica SalesOrder model
- Added functionality to populate the Extended Price for line into the Acumatica connector during order submit
- Added ShipVia mapping to the Acumatica Order Submit Job
- Removed the function to set the Commisionable Flag to False from the Acumatica Connector
- Added the fields of Overshipment/Undershipment with hard-coded values to the Acumatica SalesOrderSubmit job
- Created the scaffolding for the CSD connector
- Added the CloudSuite Distribution (CSD) connector settings to the Admin Console
- Added real-time calls for CSD
- Added support for system lists to the CloudSuite refresh processor
- Added messaging for errors on CSD integrations
- Updated P21 real-time pricing to handle bad responses from the API, because it previously failed for everything on PLP if a single item failed
- Added a real-time API connection via ApiClientCredentialsEndpoint to the P21 connector
- Updated the P21 connector default jobs
- Created the ERP connector for IFS Cloud
- Updated Inventory for IFS Aurena to include AvailableQty
- Updated inventory logic for IFS Aurena to treat an empty object as 0, not as no result
- Added new logic to the FACTS connector for Billtos that are ShipTos in the Tax/Order Submit calls
- Updated FACTS integration jobs and added Sproc for calculating order history
- Added settings to enable and configure FACTS ERP connector (documentation coming soon to Connectors for ERPs)
- Modified the integration setting replacement logic to accommodate ODBC connection for FACTS ERP
- Updated the FACTS connector to treat 0 quantity as 1
- Updated the FACTS connector to stop sending variant parents for pricing and start sending child items with their Units of Measure
- Updated the A+ real-time inventory calls to include the customer number and ship-to number and to check for duplicates and sorting (see Overview of Connectors for A+)
- Integration Jobs
- Added .csv file support to the S3-to-S3 Processor
- Added a generic Integration Connection Type and flat file processor of "AmazonS3FlatFile"
- Created a new Integration Processor, Odata Refresh, designed specifically to retrieve OData
- Moved Insite.WIS.Broker into the public surface area to expose IIntegrationProcessor
- Created a new SX.e/CSD API Integration Processor called CSDApiRefresh
- Created S3DirectUploadPreprocessor to send S3 credentials to jobs run on the WIS for use in file upload jobs
- Updated the S3 file processor to work with the Admin API and S3 Preprocessor
- Added the ability to pre-fill integration processors when there is no WIS
- Added APIROPCEndpoint connection type and support in ODataRefresh integration processor
- Changed the serialization for JSON objects from the OData Refresh processor
- Updated SQL call to handle exceptions during the GetIntegrationJob process
- Added the ability to limit integration jobs to existing records only
- Made the integration job duration (elapsed time) display more readable
- Implemented additional resiliency if jobs are set as recurring so they always get rescheduled when marked as recurring and start/end dates are in scope
- We are implementing additional resiliency into the integration platform so that if jobs are set as recurring, they will always get rescheduled as long as they are marked as recurring and the start/end dates are in scope. This is a change from the past, so please check your recurring jobs to ensure they will not get started again when you didn’t intend them to be. Currently, if you set up a job as recurring and delete the next queued job, it’s effectively stopped, but this will no longer be the case. Additionally, if a job is running when the integration service is recycled, it will fail the job as normal, but it will now honor the actions on failure, meaning if there is a linked job, that job will be run and, if a notification email is to be sent, that will also happen.
- AD Data Feed
- Added a job step to the AD Data Feed Processor for importing 360 Degree image files
- Added a job step for Attribute/Category Assignment to the AD Data Feed Processor
- Created a processor for the AD Data Feed to consume a list of URLs from Amazon S3
- Updated the base AD Data Feed processor and S3-to-S3 processor to allow overriding of asset directory
- Added a step to the AD Data Feed job to allow assignment of Products to Categories with separate Product Refresh
- Updated the AD Data Feed to read Enterworks export files
- Updated the AD Data Feed so large/small numbers are not truncated to scientific notation
- Added a new AD Data Feed Processor for Enterworks
- Windows Integration Service (WIS)
- Updated the WIS installer codes for 4.6.0 with new GUIDs for product id and upgrade code for X64 and X86 builds (See Install the WIS and Configure the WIS)
- Updated the internal WIS to load custom integration processors from the Extensions.dll file
- Modified the WIS installer to require .Net framework 4.8
- DDS Feed
- Added the Enable DDS Feed Integration setting to the Admin Console to create the DDS Feed jobs (see Connectors for the Distributor Data Solutions (DDS) Data Feed)
APIs/Handlers/Pipelines
- Changed the Settings and Spire PageByUrl APIs to return JSON format instead of XML
- For details on the Product API v2, please read Upgrade Considerations for Product API v2
- Updated the Categories API to add a new parameter of includeStartCategory to ensure the categories returned start from the startCategoryId category
- Created an Admin API to send "corrected" filenames in S3
- Made the admin/switchcms endpoints safer to address security concerns
- Injected the type (vs interface) in Products API v2 to reuse the private methods where most of the code is in CatalogPathBuilder
- Updated CallOnError and CallOnSuccess handler names to ExecuteOnErrorCallback and ExecuteOnSuccessCallback
- Implemented sort by price for real-time pricing in v1 Product API
- Updated the APIs to support grouped attributes based on the label field in the audit process
Technology and Other Changes
- Upgrade of Front-End to Angular 1.7 - Our reference themes and the Admin Console have been upgraded to the latest version of Angular.js. This was done both because that version is in long-term support and it removes some issues raised by penetration scans. We used a bridge to facilitate you not having to change and upgrade your site during this change, but we were not able to test every possible nuance, so we advise either upgrading your site to the latest version of Angular.js and doing a thorough test or, at a minimum, do a quick review of the UI on the site. This article explains the steps to upgrading it.
- Pipelining the Index Search Builder Setup - This was done to enable additional fine-grained control over how search can be extended. See this documentation.
- Unity Container Upgrade (No Longer Allowing Extension) - When we created the whitelist of packages, we included all of Microsoft packages referenced in our internal projects and did not realize that Unity was included in that list. We never expected anyone to be trying to override use Unity directly. We recently upgraded the Unity version and have discovered a handful of sites that are directly referencing the old version in their Extensions.dll. We are disabling removing Unity from the whitelist of packages, so if you have that referenced, you will no longer be able to do so. We have not yet found a situation where it was actually needed to be directly extended. Common uses and fixes are:
- Replace usages with Insite.Common.Dependencies.DependencyLocator
- CommonServiceLocator.Current.GetInstance -> DependencyLocator.Current.GetInstance
- Use a standard foreach
- Make your own ForEach extension
- IServiceLocator from the Microsoft.Practices.Unity DLL
- EnumerableExtensions.ForEach from the namespace Microsoft.Practices.ObjectBuilder2 out of the Microsoft.Practices.Unity DLL.
- Added "X-Content-Type-Options" HTTP header
- Added the Schema.Net nuget package to our standard code and whitelisted it for use
- Removed General Public License (GPL) references from our Licenses file and related screens
- Added a Google Tag Manager customer data Layer to allow for extension and merge with standard code
- Whitelisted .woff2 file extensions (in web.config) to allow Cloud customers to use these fonts
- Removed the Ruby dependency for local development
- Updated the Docker build to disable app pool recycling
- Added support for new SMTP commerce/multi-email domain feature in containerized environments
- Updated the Nuget Packages for Azure
- Updated the FileProviderFileSystem.cs to allow for support for network files
- Added PGP encryption and Azure storage to allowed Nuget packages
- Removed the call to CreateIfNotExists for the container from the Azure storage provider
- Removed SSL3 from ServicePointManager.SecurityProtocol in the base code as this is outdated and should not be used
- Added three Gen2GarbageCollection-related settings to appSettings.config to help manage memory usage
- Updated the references of HttpClient to the new HttpClientProvider
- Implemented a mechanism to disable caching on a specific page to allow use of AntiForgeryTokens
- Removed the update to SearchLookup in the Product table to eliminate unnecessary overhead
- Added Prometheus around the GeoIP call for traceability and monitoring purposes
- Obsoleted HTTPS mode
- Created a utility to programmatically extract text from a PDF file stored in the Media Library
- Obsoleted existing TokenEx settings and replaced them with new settings
- Updated routing and initilization code so it doesn't require integration/identity to have URLs in website table
- Added code to ignore translation dictionary.txt after it's generated so it isn't stored in source control
- Removed the EnableContentUrlRouteWebsiteCache app setting that was used for microsites in Cloud
- Created a pattern for converting many:many tables to full entities
- Converted many:many tables to full entities so they can be integrated with directly instead of through child collections
- Added ASP lifecycle events to the Application Log
- Removed the x-powered-by header and some related items to improve security
- Updated the AWS SDK version from 2 to 3
- Converted the RestrictionGroup rebuild to use a SQL bulk copy
- Updated image on Product List pages to use the medium product image instead of the small product image
- Added DefaultSortProperty to the PriceMatrix Form to improve performance when millions of records
- Updated code to support X-Forwarded-Host in the ActualUrl extension method
- Switched build to use npm ci instead of npm install to enforce package-lock.json commits
- Updated the nuget package to ignore System.Net.Http reference that moved to .NET SDK
- Upgraded the media library (CkFinder) to version 3.5.1.1