inRiver (IR) and Configured\_Commerce integration strategy

16 min

IR and Optimizely {user["product-name"]} have worked together to understand the data structures within IR and have arrived at the strategy for integration which will leverage the out of the box XML Schema exports provided by IR to create a data feed for {user["product-name"]} to pick up and transcribe.

Both IR and {user["product-name"]} have ETL mapping tools and the ability to write code specific to a connector (IR) or plug-in. Optimizely has taken on the responsibility to write IR-specific plug-ins and leveraging the standard schema export from IR.

{user["product-name"]} will pull Products (including Items and SKUs discussed later), Channel Nodes (category hierarchy in {user["product-name"]} parlance), and Resources (digital assets) as part of the initial, standard integration.

This document will address some schema standards we wish to impose for any IR/{user["product-name"]} customers so that we can use a standardized strategy to map IR data to {user["product-name"]} structures.

IR entity setup

We request that the following entities be required and statically named.

Channel

This is the typical publication channel that would be established for inRiver. It can represent print or other recipient. Each targeted website that would have a catalog hierarchy should be set up as a separate channel.

Channel Node

This entity represents a hierarchical representation of products within categories for a given channel/web site. This will be used to create the category hierarchy in {user["product-name"]}.

Products

We will expect a Product entity to be established that can act as the end item to be created or as a placeholder for a product that is merchandised but has variants. If there are no variants, the product record itself can be set up with the SKU number or a singular item can be set up that holds the SKU number.

Items

These are variants for the product. It will hold information specific to the variant (that is color images or specific attributes). Attribution will mostly come from the Product entity. Within Items, we will have an optional XML field called SKUs which can hold additional information about sub-level skus with their variant-specific data.

General Reasoning

inRiver will want entities that represent saleable items that capture relevant merchandising information. For example, if we have a specific shirt - we merchandise all the information about fabric, care instructions, and so on at the Product entity. We create Items for the different colors but these are not specific SKUs. At the SKU level, we have the actual sku and the size so that, together, the shirt is ultimately defined by the Product (shirt), Item (color) and SKU (size).

{user["product-name"]} entities

Product

{user["product-name"]} has a product table that can hold generic products (the same as the IR product), specific SKU's tied to a product (Items), services, add-ons, and so on. This is the master holding entity for anything that is sold or merchandised on the platform. Products that have children are termed "Styled Products".

Style Class

This entity represents a type of product with a predefined set of attributes that define it for purchasing purposes (not merchandising purposes). For example, a T-Shirt may always have a size and color but nothing else. Any number of styled products can use the same style class. This is not unlike the fieldset construct in IR.

Style Trait

This entity holds the specific traits within a style class.

Style Trait Value

These are the individual values available for the trait associated to the class.

Attribute Type

This is a more generic set of attributes and is used for product comparisons and faceting. Attribute Types are global in nature and can be assigned by category. Products then are assigned specific values for any attribute type that is associated with all of the categories the product is assigned to. CVLs in IR will be mapped to attributes in {user["product-name"]}.

Attribute Value

This is the actual value of the attributes. Attributes in {user["product-name"]} look a lot like CVLs in IR.

General Note

Currently, there is a distinction between Style Trait Values and Attribute Values even though they could be repeated within a single product. Style Traits are used to narrow down the selections of a product to the orderable SKU, while attributes are used for merchandising, searching, and comparing.

Field mapping

Rather than impose any specific naming conventions (other than entity naming conventions), there are some fields that {user["product-name"]} will require in order for the integration to be effective. The tables below will describe these requirements. The assumption is that these data will be provided in the schema and it will be up to the {user["product-name"]} or IR implementer to create the mappings during implementation.

Required fields

Target Table	Target Field	Usage/Notes
ChannelName	Name	We need to ensure that the channel name in IR matches the website name in `<>` for things to match up - there is a workaround to this shown below by using a website configuration.
Product	ERPNumber	`<>` requires a unique field to identify each product and, if this is a purchasable product, then it must also conform to the product identifier within the ERP system - this is often thought of as the SKU. For a product that is not directly orderable (styled product), there still needs to be some unique identifier.
Product	URLSegment/Name	These can be the same as the ERP Number or different but are required for all products in `<>`
Item	ERPNumber	We will still require a unique field here since items and products will both have the same target table. If the final ERP numbers are represented within the SKUs field, then that will become the ERP number and nothing more specific is required on the item itself.
Item	URLSegment/Name	These can be the same as the ERP Number or different but are required for all products in `<>`
StyleClass	Name	We will need something to anchor to in order to set up a style class. We will used a mapped field for this instead of the fieldset.
AttributeType		CVLs must be named either the same in the entity or prefixed with the entity name in order to be resolved. For example, if we have a CVL called Color, the field mapped to it in IR must be either Color or ProductColor (or ItemColor) or it cannot be mapped. The XML export does not provide enough detail to map it back to the source data element so this convention-based approach must be used.

How it is put together

We have established a series of predefined integration jobs and plug-ins to handle the integration. Each of these will be explained below along with supporting technical information regarding settings.

Because we wanted to keep this integration as standard as possible, we are using the out of the box inRiver.Connectors.Schema.Output.Files connector with a single file per entity export from inRiver. Developers can further enhance or extend this as desired - the provided code will definitely allow for a solid and deep integration but may not accommodate every circumstance given the highly configurable nature of a product information management solution.

Keep in mind that in inRiver, there are Product and Item entities where in {user["product-name"]} they are all considered products and if there is an item related to a product, those are considered styled children. The steps involved in the integration are laid out intentionally to optimally fill out all of these relationships in a particular order to complete the structures.

Warnings/Known Issues

The system does not handle multi-value fields out of the box

The system does not handle locale strings - most data fields in {user["product-name"]} are not enabled for multi-language support. The system will pull out the entry that matches the default language identified for the site. This means that we could import multi-language to language-specific sites but not to multiple languages on the same site.

While the system will create the entire category structure from the channel node feed from IR, the attributes associated with those channels is not mapped by the integration. In order for attributes to be assigned to products, the product must be assigned to a category that has those attributes assigned to it. Nothing in the integration will create that particular relationship. We suggest that the users set up the taxonomy in IR, import just the taxonomy and CVLs and then manually map the attributes to the categories before importing products and items - that way the attributes for the products will have the ability to be mapped in.

As data is changed in IR, it automatically creates XML files in the export directory. Oftentimes it will create multiple versions of the same data. The system is built to pick up these files in time-sequenced order so that the most current version will be transmitted. If an entire set of files is missing, however, the integration job will register that job as CompletedWithWarnings and you will notice in the job log that there will be warnings such as "No files found matching 'xxxxxxx' - this is a standard feature of flat file imports.

The FileUpload job is an Execution job but it still must have at least 1 step as a dummy step in order to trigger the integration processor.

If an item is modified after initial load, it generates its XML file as 'Updated' vs. 'New' and does not include the links to the parent - this breaks the integration for that item. We are working with inriver to have them fix that behavior.

Technical extensions

Other than the integration plug-ins defined below, the following are some special constructs used to support the integration.

WebsiteConfiguration. There is only one special configuration entry that is generated automatically provided that the channel name in IR matches the website name in {user["product-name"]}. The system will create an entry called inRiver Channel EntityId which contains the IR internal entity id # which is a simple identity/integer field. This is used in different places in the system to tie data together from the

InRiverCommonVocabulary table. Since CVL-linked data in products and items don't resolve to their value but only send up their entity id, we are persisting that data directly on the server in a special table built by the custom post processor.

Integration jobs and plugIns

Things to understand

In general, there are a number of fields that come back from the integration processors that are "free". There are other fields, primary the ones in the Product and Item entities that only come back for mapping if they are included in the select statement. All fields within the XML structure "ProductFields" or "ItemFields" are available in this manner.

The UniqueId field is a special field and is assigned and should be mapped to the ERPNumber field (natural key in {user["product-name"]})

PlugIns

Name	Type	Usage
InRiverLookupInformation	Preprocessor	This is used to send the contents of the CVL table back into the integration processors to resolve for data values
InRiverCommonVocabulary	IntegrationProcessor	Gathers all of the information for CVLs and places them into a predefined dataset formal for consumption on the server
InRiverCommonVocabulary	Postprocessor	Consumes the predefined CVL format and creates Attribute Types, Attribute Values and the translations for all of the values
InRiverProduct	IntegrationProcessor	Gathers and formats the data from the Product entity including the relationships, key information, and specified fields from the selection statement
InRiverStyleClass	IntegrationProcessor	Gathers and formats the data for Product to Item relationships and formats in a way that can be consumed for populating Style Classes, Style Traits and Style Trait Values
InRiverItem	IntegrationProcessor	Similar to the product version, this gathers information about the item entity
InRiverChannel	IntegrationProcessor	Gathers and formats the data for the channel in order to create the category hierarchy
InRiverChannelNode	IntegrationProcessor	Gathers and formats the data for the channel node to create the category hierarchy
InRiverResource	IntegrationProcessor	Gathers the information so that the standard images can be mapped into the product table

Integration jobs

Below are the details of the various jobs that we created to help enable and streamline a given integration from InRiver to {user["product-name"]}. These are starting points for the most part and will need to be adjusted to meet any client's specific needs.

We build these jobs where possible to implement the standard field mapper so that the implementation consultant would be able to add new fields and put them where they way when integration with IR.

Some of the integrations have some fields available as "free fields" to use that do not need to be included in the selection column list. These are as follows:

Channel

UniqueIdentifier

ChannelNode

UniqueIdentifier

CategoryName

ProductEntityId

Item

UniqueIdentifier

Active

ParentProductId

Product

UniqueIdentifier

Active

Resource

UniqueIdentifier

SmallImagePath

MediumImagePath

LargeImagePath

StyleClass

StyleClass

StyleTrait

StyleTraitValue

Job definitions

Name/Step	Processors (prefixed with 'InRiver') or Object	What it does/Notes
Common Vocabulary Refresh	Pre: LookupInformation WIS:CommonVocabulary Post:CommonVocabulary	Reads all CVLs and creates the internal table, creates/updates AttributeTypes, Attribute Values and all translations of them This pulls from files that are masked with CVLS/.xml.
Step 1 - Attribute Types	AttributeType	This job should not be altered.
Step 2 - Attribute Values	AttributeValue	This job should not be altered
Step 3 - Translation Dictionary	TranslationDictionary	This job should not be altered
Product Refresh	Pre: LookupInformation WIS: Product Post: FieldMap	This job will populate the style classes, traits, values and create/update the product record for product entities This pulls from files that are masked with _Product_.xml
Style Class	StyleClass	Pick a product field to use as the name of the style class - if you don't have one, you will likely need to add one. The field map should only need to list the fields that define the style class code/name and description (which can be the same as the name field). The sample set uses a field called ProductGroup which may or may not exist in your dataset. The field map is to the object StyleClass and is used to create the actual classes
Product Initializer	StyleProductInfo	The intent of this step is simply to establish the existence of the parent/styled product in the database and then relate it to the style class it belongs to. The rest of the relationships happen in the Item refresh. Note: You must populate the ERPNumber, Name and URLSegment from the data. ProductGroup and ProductNumber are proxies for the fields you would use in your specific implementation of inRiver.
Product Finalizer	Product	This is the step that will fill out all of your information for a product including the basic content. We recommend that you have a short description field specifically available. Only the fields listed in the Columns tab will be pulled over and they must exist in the ProductFields section of the XML. Then you can map them to product content, product custom properties or directly to product fields.
Product Attributes	Product	While this does not need to be a separate step, it is separated out for clarity only. For this to work, you select the product fields you want mapped as attributes - these are limited to CVL fields in IR. For each value selected, you set up a ChildCollection where the From Property is the field from IR and the ToProperty is AttributeValues. Note:These will only map if the attribute types are named specifically the same as the field name or the field name with the entity prepended. The attribute types must already be mapped to one of the categories associated with the product or it also cannot be successfully mapped.
Item Refresh	Pre: LookupInformation WIS: StyleClass Post: FieldMap	This job will create the products for the child products and then update the style traits and values. This pulls from files that are masked with _Item_.xml
Style Trait Refresh	StyleTrait	The values you must pull out of the Item entity need to be prefixed with 'Variant' to identify the actual style trait being used. For example, VariantColor will map to style trait Color. Note that in `<>`, style traits are not the same as attributes - they are similar and may actually be the same values but are used in different ways. The column list only needs to be a list of the variant values to map. StyleClass is a "free" column that will map the item to the parent item's style class. The mapping should not be altered - we need the StyleTrait Name and Class only - description is optional and is typically set to the same as the name. This step will create all of the unique combinations of style traits associated with the style class based on the items sent from IR.
Style Trait Value Refresh	StyleTraitValue	The column list here should be the same as for the style trait step. This steps will create the actual valid values within each style trait based on the items sent from IR. The mapping should not be altered.
Item Refresh	StyleProductInfo	This is a bit more constrained object - not all properties are available to it. This step will create/update the products that are styled children in B2B and set up the associated style trait values to that product. This step must set the ERPNumber, Name and URLSegment fields. The query must contain all of the variant fields. Additional fields like short description or any other ItemField would be mapped into this step as well.
Category Refresh	Pre: LookupInformation WIS: Channel Post: FieldMap	This pulls from files that are masked with _Channel_.xml
Channel Refresh	WebSiteConfiguration	All this step does is establish the WebsiteConfiguration entry "inRiver Channel Entity" to implant the unique identity in order to tie the website to the channel. This mapping should not be altered.
Channel Node Refresh	Category	This step creates/updates categories and their hierarchy and assigns products to the categories. It is unlikely this will require much change in the mapping.
Resource Refresh	Pre: LookupInformation WIS: Resource Post: FieldMap	This pulls from files that are masked with _Resource_.xml
Resource Refresh	Product	This step will embed the path of the resources (images) into the product for SmallImagePath, MediumImagePath, and LargeImagePath. The mapping should not be changed. There is a step parameter called inRiverResourceLocation that is used to identify the prefix to the actual name of the file. The default in the job is /UserFiles/Images/inRiver. The original images are of any type and the full name is provided in the interface from inRiver, however the smaller images are automatically resized and renamed using configuration within inRiver. To accommodate this we have the PreviewExtension and ThumbnailExtension parameters that will define the extension with a default of jpg. The plug-in will map the Thumbnail image to SmallImagePath, Preview image to MediumImagePath and Original to LargeImagePath.
FileTransfer	Pre: None WIS: FileUpload Post: None	This is a special purpose job that is used to physically copy the files from the source location into the target location. The WIS uses a web service to move the images. This is an execution job which uses no steps but has to have at least 1 step in order for the WIS processor to pick it up. The presumption is that the publishing process in inRiver will populate the resource files with the images based on changes made to the entities. The following parameters control the process: SourceDirectory - the location on the inRiver server where the resources are located SourceMask - the mask used to define the files to select and send and can include multiple masks separated by a pipe (\|) Destination Directory - the relative location on the B2B server to place the images - default value is images/inriver. DeleteSourceFilesOnCopy - true or false - once copied up, indicates if the files should be deleted on the IR server