Migrating VPP-based files to the new media system

13 min

In Optimizely Content Management System (CMS) 7.5 a new media file system with a new user interface was introduced. Content stored in this system is referred to as “media”. This system is based on the same content system used for blocks and pages, as well as products from the catalog in Commerce. This means that files have support for routing, custom properties, typed models, custom renderings, security and more. A BLOB provider has also been added to the CMS platform, to make it easy to change storage model for media. Moving existing sites to content-based file management requires a few manual steps outlined in this document.

Note Run the VPP Migration tool after the site has been upgraded to CMS 7.5 via Deployment Center. Running migration on sites upgraded via NuGet to later versions is not supported.

Background

Prior to this version, files in CMS (as exposed in the old file manager) were handled by a VPP (Virtual Path Providers) system. This is an ASP.NET feature to expose for example an aspx file to ASP.NET from a provider. On top of the VPP system a layer called “Unified Files” was added to bridge the VPP system into EPiServer CMS. Files that are shown in the CMS File Manager must implement this Optimizely-specific API. The core VPP system will continue to work as before in EPiServer to allow easy integration with ASP.NET, and the aging “Unified Files” bridge will be phased out in favor of content-based files.

Note The VPP-based file management and the content-based file management are both supported from an API perspective in this version to enable migration as a separate step, but the user interface only supports the new media file system.

Preparation

Note Make sure the site is upgraded using Deployment Center before you start the migration process.

The following data will be migrated:

Folder structure

Metadata

Version history of files will not be migrated.

Determine if the site is using metadata for files and define a base class containing those properties. Metadata was by default defined in the file FileSummary.config in the root folder of the site, but on many sites this metadata is not used at all. Property named Category is not supported.

Below is an example code that will migrate over the Description property from the default metadata on all files and separate types for images and video. The base classes ImageData and VideoData used in the sample code provides some additional features in the user interface such as thumbnails for images and the ability to specify pickers that selects for example images (using UIHints in code).

[ContentType(GUID = "EE3BD195-7CB0-4756-AB5F-E5E223CD9820")]
public class GenericMedia: MediaData {
  public virtual string Description {
    get;
    set;
  }
}

[ContentType(GUID = "0A89E464-56D4-449F-AEA8-2BF774AB8730")]
[MediaDescriptor(ExtensionString = "jpg,jpeg,jpe,ico,gif,bmp,png")]
public class ImageFile: ImageData {
  public virtual string Description {
    get;
    set;
  }
  public virtual string Copyright {
    get;
    set;
  }
}

[ContentType(GUID = "85468104-E06F-47E5-A317-FC9B83D3CBA6")]
[MediaDescriptor(ExtensionString = "flv,mp4,webm")]
public class VideoFile: VideoData {
  public virtual string Description {
    get;
    set;
  }
}

Migrating data

The next step is to start the migration. Make sure you have a running site with the above file types defined before you use the migration tool.

Start the Migration Tool.

Select the path to the EPiServer site and click Connect.

Note The site will now start in a .NET AppDomain hosted by the migration tool.

On the left side select the virtual path providers to migrate. The file types found are displayed to the right.

Keep the default settings and click Migrate to start the process. This process may take a while depending on the number of files in the system.

Final changes

The Migration Tool automatically removes the configuration for the migrated virtual path providers from the episerver.framework section in web.config. If you have not made any changes to this section after installing EPiServer, then the migration tool removes these providers: Page Files, Global Files and Documents. Note that leaving these providers will cause problems with links on the site since the new content-based files will have the same identify as the VPP-based files.

The new files are stored in a subfolder Blobs based on the path attribute on the appData element in the episerver.framework section in web.config (if you are using the default file based BLOB provider). The old files are not deleted automatically from disk, you can safely remove them after the migration has been completed (by default they are stored folder folders Documents, Global, and PageFiles).

The site should now be fully operational with links pointing to the new content-based files and you should have a new user interface for working with files in the editorial interface.

Custom virtual path providers

Custom virtual path providers will not work after migration when the data is migrated over to a new system. However, the migration tool does support migrating data from custom providers into content-based files. The sections below provide guidance on provider extensibility for content-based files.

Storage integration

A content-based file supports a new model called BLOB providers, which allows you do changes where the actual binary stream is stored but not where metadata is stored. A BLOB provider is a very small API to make it simple to create optimized storage providers for different hosting environments.

Structure integration

The content structure for files supports content providers, as any content structure does. It is required to make a content provider when metadata is provided from an external system.

Export/Import

If you export data from a site with VPP-based files, and import the data to a site with content-based files, the files will be automatically converted to content-based files. All none-page files will be created under the Global Media Root. See Known Limitations for more info on native virtual path providers.

Changes in API

If the site uses the Virtual Path Provider API, then you need to change your code as well, since most sites do not use this API including the Alloy sample site in EPiServer 7. The old API will work without throwing any exceptions, but will not return any files after you migrated. The old API was accessed using the ASP.NET API HostingEnvironment.VirtualPathProvider. Examples of using content-based files are available in the Developer Guide under the Content section in the CMS SDK.

Redirect old paths to new media paths

During migration a table _MigratedVPPFiles is created in the database. This table is a lookup table containing:

VirtualPath – The original path in the VPP system

ConvertedDate – The local date and time when the file was converted

ContentLink – The content link of the new media item

ContentGuid – The content guid of the new media item

The purpose of this table is to make it possible to lookup and map paths after the migration has completed. The following sample code uses this table to redirect from old VPP paths to new media paths:

using EPiServer;
using EPiServer.Core;
using EPiServer.Data;
using EPiServer.Framework;
using EPiServer.Framework.Initialization;
using EPiServer.ServiceLocation;
using EPiServer.Web;
using EPiServer.Web.Routing;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.Caching;
using System.Web.Routing;

namespace VppToContentRedirectModule {
  /// <summary>
  /// Automatically redirects old VPP paths registered by the migration tool to new media paths
  /// </summary>
  [ModuleDependency(typeof (EPiServer.Web.InitializationModule))]
  public class VppToContentRedirect: RouteBase, IInitializableModule {
    private string[] RootPaths = new string[] {
      "~/Global/",
      "~/PageFiles/",
      "~/Documents/"
    };

    public override RouteData GetRouteData(HttpContextBase httpContext) {
      if (httpContext == null || !VirtualPathUtilityEx.IsValidVirtualPath(httpContext.Request.Path)) {
        return null;
      }
      var appRelativePath = VirtualPathUtility.ToAppRelative(httpContext.Request.Path);
      if (!RootPaths.Any(p => appRelativePath.StartsWith(p, StringComparison.OrdinalIgnoreCase))) {
        return null;
      }
      return new RouteData(this, new VppRedirectRouteHandler());
    }

    public override VirtualPathData GetVirtualPath(RequestContext requestContext, RouteValueDictionary values) {
      return null;
    }

    private class VppRedirectRouteHandler: IRouteHandler {
      public IHttpHandler GetHttpHandler(RequestContext requestContext) {
        return new VppRedirectHttpHandler();
      }
    }

    private class VppRedirectHttpHandler: IHttpHandler {
      const string CacheKeyPrefix = "VppToContentRedirect-";
      const string MigrationToolLookupSql = "SELECT ContentLink FROM [dbo].[_MigratedVPPFiles] WHERE VirtualPath=@p0";

      public bool IsReusable {
        get {
          return true;
        }
      }

      public void ProcessRequest(HttpContext context) {
        var vppPath = context.Request.Path;

        var cacheKey = CacheKeyPrefix + vppPath;
        var newUrl = context.Cache[cacheKey] as string;
        if (newUrl == null) {
          var db = ServiceLocator.Current.GetInstance < IDatabaseHandler > ();
          var contentLinkString = db.Execute(() => {
            using(var cmd = db.CreateCommand(MigrationToolLookupSql, System.Data.CommandType.Text, vppPath)) {
              return cmd.ExecuteScalar() as string;
            }
          });

          if (String.IsNullOrEmpty(contentLinkString)) {
            throw new HttpException(404, "Not Found");
          }

          ContentReference contentLink = new ContentReference(contentLinkString);
          try {
            newUrl = ServiceLocator.Current.GetInstance < UrlResolver > ().GetUrl(contentLink);
          } catch (ContentNotFoundException) {}

          if (String.IsNullOrEmpty(newUrl)) {
            throw new HttpException(404, "Not Found");
          }
          context.Cache.Insert(cacheKey, newUrl, DataFactoryCache.CreateDependency(contentLink), Cache.NoAbsoluteExpiration, TimeSpan.FromHours(1));
        }

        context.Response.RedirectPermanent(newUrl);
      }
    }

    public void Initialize(InitializationEngine context) {
      if (context.HostType == HostType.WebApplication) {
        RouteTable.Routes.Add(this);
      }
    }

    public void Preload(string[] parameters) {}

    public void Uninitialize(InitializationEngine context) {}
  }
}

Known limitations

By default CMS uses the VirtualPathVersioningProvider as the provider for files, the following applies for sites using VirtualPathNativeProvider and other providers not implementing permanent links.

Native virtual path providers can be migrated but references to native providers are stored in content (for example in an XhtmlString) as they appear for visitors, for example, /Global/Folder/File.jpg (compared to for example VirtualPathVersioningProvider where the reference would be stored as ~/link/CD402716B038496E95E4EA6D24AB497B.jpg, called a permanent link). The Migration Tool will not parse through all content and change the references to permanent links. Existing links in content will be broken after the migration (since the generated content-based file will get a different URL) unless the redirect example above is deployed.

If you export data from a site with files from a native virtual path provider, and import the data to a site with content-based files, the files will be automatically converted but the import will not update content to use permanent links causing broken links, the above mentioned mapping table will not be updated in this case so links have to be manually changed.

Obsolete features

The new content-based files does not support WebDAV.

VPP migration tool (requires log in to Optimizely World)