Map stores

13 min

Compile time type mapping

When instances of a compile time data type (.NET classes excluding EPiServer.Data.Dynamic.PropertyBag and classes implementing System.IEnumerable) are saved in the Dynamic Data Store, and their "inline" properties are mapped to columns in the "big table." This is known logically as a store.

Default mapping

The default algorithm for mapping .NET classes (excluding EPiServer.Data.Dynamic.PropertyBag and classes implementing System.IEnumerable) to a store is as follows:

Property must have a getter and setter

Property must be marked public (although the setter can mark non-public if desired)

Other properties are ignored and not saved in the Dynamic Data Store

Custom mapping

You can override the default mapping behavior. This is useful if you do not want certain public properties to be mapped or want certain non-public properties to be mapped.

To use custom mapping, you need to add the System.Runtime.Serialization.DataContactAttribute to your class definition. In this case, only properties marked with the System.Runtime.Serialization.DataMemberAttribute are mapped and saved in the Dynamic Data Store regardless of the accessibility status. They must still, however, have a getter and setter.

See the MappingWithDataContract class in the DDS sample project.

You may want to save an object of an existing class that has already been marked with DataContactAttribute and its member properties with DataMemberAttribute. One problem might be that using these properties does not match the desired behavior when an object instance is saved in the Dynamic Data Store. In these cases, you can also add the EPiServerDataContractAttribute to the class definition and EPiServerDataMemberAttribute to the properties to be saved. The Dynamic Data Store will use these attributes in preference to the Microsoft ones to resolve the conflict.

See the MappingWithEPiServerDataContract class in the DDS sample project.

Type handlers

Some classes in the .NET Framework do not have properties that the Dynamic Data Store can use to extract the value, save it to the database, and re-inflate an instance with the value from the database. If you want to use such a type as a property on a class that is saved to the Dynamic Data Store, you must register a Type Handler for it.

A good example of this is the System.Uri class. This class only has read-only properties, and saving an instance to the Dynamic Data Store without a Type Handler is meaningless as no data is stored for it.

See the MappingWithTypeHandler class in the DDS sample project

Map runtime data type (PropertyBag)

Properties saved using PropertyBags are mapped as if the properties were public members' properties on a normal .NET class. PropertyBags can be mapped in the following ways:

Implicit is the first time a PropertyBag is saved to a store, the store mappings are inferred from the properties in the PropertyBag.

Explicit is a mapping dictionary which is passed to the DynamicDataStoreFactory.CreateStore method detailing the store mapping. The main advantage of this is that properties that can be potentially saved in this store are mapped, as opposed to the first time a PropertyBag is saved, which may not have all potential values.

See the ImplicitDynamicMapping and ExplicitDynamicMapping classes in the DDS sample project.

Store remapping

From time to time, you may need to change the structure of your data. This can mean adding, removing, or changing properties on your .NET classes or PropertyBags.

The Dynamic Data Store is flexible regarding accepting changes to types saved in a store.

You can remap .NET classes to stores in the following ways:

Using attributes on the class.

Using attributes with the StoreDefinition class. Remapping stores that are not represented by a .NET class can only be done with the StoreDefinition class.

Remap using class attributes

A .NET class whose instances are saved in the Dynamic Data Store can be optionally decorated with the EPiServerDataStoreAttribute. In these cases, set the AutomaticallyRemapStore property to true. When the Optimizely application starts, it scans for classes with this attribute and automatically remaps the .NET class to the store if needed. Any properties that are renamed MUST be marked with the EPiServerDataPropertyRenameAttribute attribute; otherwise, the remap treats them as if one property was removed (with the old name) and one was added (with the new name).

Warning In multi-instance environments (such as Azure App Service or load-balanced servers), AutomaticallyRemapStore = true can cause transient Invalid column name errors when you add, remove, or rename properties. See Multi-instance deployment considerations.

Remap using StoreDefinition class

To remap a store, obtain the store definition of a store with the StoreDefinition property of a DynamicDataStore instance or with the StoreDefinition.Get method.

You can then call the Rename and Remap methods to update the store's mappings. You should call Rename before Remap. Otherwise, properties renamed in the data type are treated as if one property was removed and added.

Finally, call the CommitChanges method of StoreDefinition to update the store's meta information in the database. If a DynamicDataStore instance reference is held, then its Refresh method should be called to align its in-memory copy of the store definition with the one committed to disk.

Remap rules

These rules are followed when remapping stores:

Properties removed from the type definition are removed from the store.

Note The data itself for the removed property remain intact in the big table. It is only the “view” of the data that is changed.

Properties added to the type definition are added to the store.

Properties with the same name but different data types are checked for compatibility. When old and new property data types are 'inline,' the database used must be able to convert from the old database data type to the new one. If a collection or reference type is changed, then the new property type must be assignable (using System.Type.IsAssignableFrom) or the old type must be convertible to the new type. The old type supports System.IConvertible and the System.Convert.ChangeType succeeds in converting an instance of the old type to an instance of the new type.

See the StoreReMapping class for examples of store re-mapping and the PropertyReName class for updating mappings when a property is renamed on a type in the Dynamic Data Store SDK.

Multi-instance deployment considerations

When AutomaticallyRemapStore = true, Optimizely applies schema changes to a Dynamic Data Store (DDS) type during application startup. Schema changes include adding, removing, or renaming properties. In a single-instance environment, this is safe because no requests are served until startup completes. In a multi-instance environment, several instances may start simultaneously during a deployment. Optimizely does not coordinate the schema change between them.

This can lead to a race condition:

One instance starts applying the schema change (such as adding a column) while another queries the same store before the change finishes.

The second instance reads the database in an intermediate state and throws a SQL exception such as Invalid column name 'RenditionUrl'.

The error typically appears 15–20 seconds after startup, as a SQL exception during a query rather than a schema update failure. It is usually transient and resolves after all instances complete the remap. However, requests served in the meantime can fail.

Environments where this applies include Azure App Service (especially with slot-swap deployments and multiple instances) and load-balanced server farms.

Pre-create stores with an IInitializableModule for multi-instance deployment

To avoid serving requests before the schema change is complete, pre-create the affected stores in an IInitializableModule. Because the module runs as part of the initialization pipeline, Optimizely creates the store and any pending remap before the instance handles requests.

[InitializableModule]
[ModuleDependency(typeof(EPiServer.Web.InitializationModule))]
public class DdsSchemaInitializer : IInitializableModule
{
    public void Initialize(InitializationEngine context)
    {
        // Pre-create stores before serving requests
        DynamicDataStoreFactory.Instance.CreateStore(typeof(YourDdsType));
    }
    public void Uninitialize(InitializationEngine context) { }
}

Use Azure warm-up for multi-instance deployment

On Azure App Service, use the WEBSITE_WARMUP_PATH application setting with a health endpoint that calls CreateStore() for the affected types. Azure requests the warm-up path before adding the instance to the load balancer rotation. This ensures the schema change is complete before the instance receives traffic.

Use rolling deployments for multi-instance deployment

Use rolling deployments so that only one instance restarts at a time. Each instance applies the schema change before the next instance starts.

Map types to specific stores

It can be convenient to save instances of a Type in the same store, regardless of where those instances are in an object graph. You have the following options:

Global Mapping – Use the EPiServer.Data.Dynamic.GlobalTypeToStoreMap.Instance methods to add and remove a mapping or add an EPiServer.Data.Dynamic.EPiServerDataStoreAttribute to the .NET class. When this is used, instances of the registered type are saved in the store with the specified name.

Local Mapping – A delegate is passed to the DynamicDataStore or DynamicDataStore<T> Save method. This delegate is called for "reference" properties and collection items that are references. The delegate should return the name of the store to save the item. This gives extra flexibility to steer instances of a Type into different stores depending on their context and use.

Note The store top level item, that is, the object passed to the DynamicDataStore Save method, is always saved in the current store (the store Save is being called on) regardless of the name of that store. This effectively overrides the Type to Store mapping mechanism.

Example:

A global Type to Store mapping is added so instances of the Type "Person" are saved in a store called "People."

A store is created or obtained with the name "MyPeople."

An instance of a Person Type is saved in the store obtained in step 2. This instance has properties that are also Person instances.

Result: The top-level Person is saved in the "MyPeople" store but other instances of the Person Type in the object graph are saved in the "People" store (because of the global mapping).

To adhere to the global Type to Store mappings, you should create or obtain your top-level store by calling DynamicDataStoreFactory.Create or GetStore with just the type and not a store name.

See the UsingGlobalTypeToStoreMapping and UsingLocalTypeToStoreMapping classes in the DDS sample project for more details.

Map stores to custom big tables

In the same way as it may be convenient to map a Type to a Store, it may also prove convenient to map a store to a custom big table. See the Big Table section in the Dynamic Data Store topic for more details.

The UsingGlobalStoreToTableMapping class in the DDS sample project demonstrates this technique.