Control field indexing

18 min

Optimizely Graph has a hard limit of 100,000 fields. Every property you sync to Graph adds one or more fields to the schema. Graph maintains each field on every content update. As you approach the limit, syncing and queries slow down. Control indexing at the property level to stay within the limit and keep publishing and queries fast.

Before you begin

Every supported CMS version has an indexing type per property, but the way you set it differs. Confirm you have the access each version requires:

CMS 12 – A code change and a deployment. The Optimizely.ContentGraph.Cms.NetCore package supplies the attribute and the conventions API.

CMS 13 – A code change and a deployment, or CMS administrator access to the Admin UI.

CMS (SaaS) – CMS administrator access, or an API client authorized to call the content type REST API.

Why field count matters

Every property on every content type that you sync to Optimizely Graph creates one or more fields in the Graph schema. Graph updates every indexed field each time the content changes. A higher field count means more work per sync and higher latency when you publish.

The 100,000-field limit sounds like a concern only for large accounts with hundreds of content types. In practice, field count grows faster than the content type and property counts suggest, so smaller accounts reach it too.

Indexing types

Not every property needs filter or search indexes. Some properties you only read in a GraphQL query. You never filter them with a where clause or use them for full-text search. Give those properties the retrieve-only indexing type. The value still returns in query responses. Graph removes only the filter and search indexes, so the property uses fewer fields against your limit.

Each property has an indexing type. The indexing type determines whether Graph does one of the following:

Stores the property and returns it in results.

Indexes it for filtering and sorting.

Indexes it for full-text search.

Leaves it out of the schema.

The behavior is the same across CMS versions, but the names differ.

Behavior	CMS 12	CMS 13	CMS (SaaS)
Retrieved in query results. Not queryable, sortable, or facetable.	OnlyStored (conventions API) PropertyIndexingMode.OutputOnly (attribute)	Default	Omit indexingType
Filterable, sortable, and facetable. No full-text search. like matches a substring anywhere in the value, and endsWith is available.	Queryable (conventions API) PropertyIndexingMode.Default (attribute)	Queryable	queryable
Filterable, sortable, facetable, and full-text searchable. Adds match and contains, drops endsWith, and like matches the start of a token instead of a substring.	Searchable (conventions API)	Searchable	searchable
Not indexed. Cannot be retrieved or queried.	ExcludeField (conventions API)	Disabled	disabled

CMS 12 has no equivalent of Disabled. To keep a property out of the schema in CMS 12, use ExcludeField in the conventions API.

Note The same behavior has different names in different versions. CMS 12 calls the retrieve-only behavior OnlyStored. CMS 13 and CMS (SaaS) call it Default. In CMS 12, Default means something else it is the queryable behavior. Check which version you are working in before you copy a value from one project to another.

Searchable is not a superset of queryable

The names suggest that the searchable behavior adds full-text search on top of everything the queryable behavior does. It does not. Switching a property to searchable changes which operators the field accepts, and changes what the like operator matches.

Adds – match and contains, for relevance-ranked full-text search.

Removes – endsWith. A query that uses endsWith on a searchable field returns a schema error.

Changes – like. On a queryable field, like matches a substring anywhere in the value, including the middle of a word. On a searchable field, like matches only the start of a token.

Important The like difference fails silently. The query stays valid and returns a result set, so a search that worked against a queryable property can return fewer results, or none, after you switch that property to searchable. Review every like filter before you change the indexing type of a field. For the operator-by-operator comparison, go to Operators.

Example of the same query on each indexing type

An article has Name set to ChangRyul hazards report. The following query filters on a sequence of characters in the middle of a word:

{
  ArticlePage(where: { Name: { like: "%zards%" } }) {
    items {
      Name
    }
  }
}

The result depends on how you indexed Name:

Queryable – Returns the article. like matches "zards" inside the token "hazards".

Searchable – Returns nothing. like matches only tokens that start with "zards", and no token in the value does.

To match the same article on a searchable field, start the term at a token boundary. like: "%hazards%" returns the article, because the token "hazards" starts with "hazards". The same rule explains why like: "%ChangR%" matches the token "ChangRyul" while like: "%Ryul%" does not.

Index a property for substring matching and full-text search

A property has one indexing type. You cannot index a single property as queryable and searchable at the same time, so no single field gives you mid-word substring matching and relevance-ranked full-text search together.

To get the two behaviors for one piece of content, add a companion field. Keep the original property searchable for full-text search, add a computed field that returns the same value, and combine the pair in a GraphQL _or filter.

Add a method to the content type that returns the value you want to match on. The following example is CMS 13, because the [IndexingType] attribute exists only in CMS 13:

[ContentType(DisplayName = "Article Page", GUID = "CONTENT_TYPE_GUID")]
public class ArticlePage : PageData
{
    // Full-text search with match and contains
    [IndexingType(IndexingType.Searchable)]
    public virtual string Title { get; set; }

    // Companion value for substring matching with like
    public string TitleSubstring() => Title;
}

In CMS 12, leave the attribute off the property. The CMS 12 [GraphProperty] attribute has no searchable mode, so make Title searchable through the conventions API with Set(x => x.Title, IndexingType.Searchable), or through the standard CMS property setting, before you register the companion field.

conventionRepository.ForInstancesOf<ArticlePage>()
    .IncludeField(x => x.TitleSubstring());

Graph indexes a field added through IncludeField as StringFilterInput, the queryable type. Because the method reads Title when Graph indexes the content, the companion field stays in sync without extra code. For more about computed fields, go to Indexing conventionsIndexing conventions.

Query the pair with _or so that a hit on the searchable property or on the companion field returns the item:

{
  ArticlePage(
    where: {
      _or: [
        { Title: { match: "hazards report" } }
        { TitleSubstring: { like: "%zards%" } }
      ]
    }
  ) {
    items {
      Title
    }
  }
}

The companion field costs an extra set of fields against your 100,000-field limit, so apply the pattern only to the properties that need substring matching and full-text search.

How Graph counts fields

The number of fields Graph creates is not the same as the number of properties on your content types. Depending on how Graph indexes a property, Graph maps that property to more than one field in the schema:

A retrieve-only property uses the fewest fields.

A queryable property uses more fields than a retrieve-only property.

A searchable property uses the most fields.

Queryable and searchable properties each use more than one field, so your field count grows faster than your property count. Reducing indexing on properties you do not filter or search is the most direct way to keep the count down.

Where you set the indexing type

The values are similar across versions. Where you set them is not.

Method	CMS 12	CMS 13	CMS (SaaS)
C# attribute	[GraphProperty]	[IndexingType], [Searchable]	Not applicable
Conventions API	Set(), ExcludeField()	Computed fields only	Not applicable
Admin UI	Not applicable	Settings > Content Types	CMS UI
Content type REST API	Not applicable	indexingType	indexingType

CMS 12

CMS 12 controls indexing in code, through the Optimizely.ContentGraph.Cms.NetCore package. Use either the [GraphProperty] attribute or the conventions API.

Add the [GraphProperty] attribute to a C# property and pass a PropertyIndexingMode. The enum has two values:

PropertyIndexingMode.Default – Stored and filterable, but not full-text searchable.

PropertyIndexingMode.OutputOnly – Stored and returned in query results only. Graph does not build a filter or search index for it.

[ContentType(DisplayName = "Article Page", GUID = "CONTENT_TYPE_GUID")]
public class ArticlePage : PageData
{
    // Default: stored and filterable, but not full-text searchable
    [GraphProperty(PropertyIndexingMode.Default)]
    public virtual string Author { get; set; }

    // OutputOnly: returned in query results only, not filterable or searchable.
    // This reduces the field-count and indexing impact of the property.
    [GraphProperty(PropertyIndexingMode.OutputOnly)]
    public virtual string InternalNotes { get; set; }
}

OutputOnly reduces overhead without losing access to the data. Graph still returns the value when you query for it. Because Graph does not index the property for filtering or search, it costs less of your indexing and query budget. Use OutputOnly for every property you do not filter or search on.

[GraphProperty] only reduces indexing. The attribute has no Searchable mode. To make a property full-text searchable, mark it searchable through the standard CMS property setting. Graph then indexes it for search unless a [GraphProperty] mode of Default or OutputOnly overrides the setting.

The conventions API configures indexing without attributes, using the singleton ConventionRepository. Call ForInstancesOf<T>() and then Set(...) per field with an IndexingType:

IndexingType.OnlyStored – Stored and returned only, not queryable or searchable. Equivalent to PropertyIndexingMode.OutputOnly.

IndexingType.Queryable – Filterable and queryable. Equivalent to PropertyIndexingMode.Default.

IndexingType.Searchable – Full-text searchable. Supported only for primitive types, string, and DateTime.

conventionRepository.ForInstancesOf<ArticlePage>()
    .Set(x => x.MainBody, IndexingType.Searchable)      // full-text searchable
    .Set(x => x.Author, IndexingType.Queryable)         // filterable, queryable
    .Set(x => x.InternalNotes, IndexingType.OnlyStored) // returned only
    .ExcludeField(x => x.AdminNotes);                   // dropped from the schema entirely

Graph neither stores nor returns a field removed with ExcludeField.

See Conventions APIConventions API for the full CMS 12 conventions reference.

CMS 13

CMS 13 moves property indexing out of the conventions API. Set the indexing type with the [IndexingType] attribute, in the Admin UI, or through the content type REST API.

Apply the [IndexingType] attribute from EPiServer.DataAnnotations and pass a value from the EPiServer.DataAbstraction.IndexingType enum:

using EPiServer.DataAbstraction;
using EPiServer.DataAnnotations;

[ContentType(DisplayName = "Article Page", GUID = "CONTENT_TYPE_GUID")]
public class ArticlePage : PageData
{
    // Searchable: full-text searchable, sortable, and facetable.
    // Accepts match and contains, but not endsWith.
    [IndexingType(IndexingType.Searchable)]
    public virtual string Title { get; set; }

    // Queryable: filterable, sortable, and facetable, but not full-text searchable.
    // Accepts endsWith and substring matching with like.
    [IndexingType(IndexingType.Queryable)]
    public virtual string Author { get; set; }

    // Default: returned in results only
    [IndexingType(IndexingType.Default)]
    public virtual string Subtitle { get; set; }

    // Disabled: kept out of the Graph schema entirely
    [IndexingType(IndexingType.Disabled)]
    public virtual string InternalNotes { get; set; }
}

[Searchable] is shorthand for [IndexingType(IndexingType.Searchable)], because SearchableAttribute derives from IndexingTypeAttribute.

The [IndexingType] attribute takes precedence over the Admin UI. When the attribute sets the indexing type in code, the Admin UI cannot change it for that property. For a content type defined in code, set the indexing type in code for every property. Keeping the whole content type in one place avoids surprises for the next developer.

For a content type that the attribute does not cover, use the Admin UI:

Go to Settings > Content Types.

Select the content type.

Set Property Indexing Type for the property.

Setting Property Indexing Type to Disabled removes the property from the Graph schema.

Note The CMS 12 conventions methods Set() and ExcludeField() do not exist in CMS 13. Migrating a CMS 12 project means moving that configuration to the attribute, the Admin UI, or the REST API. The CMS 13 conventions API accepts an IndexingType, but only for computed fields you add with IncludeField() — not for existing CMS properties.

See Indexing conventionsIndexing conventions for computed fields, and Migrate the Conventions API from CMS 12 to CMS 13Migrate the Conventions API from CMS 12 to CMS 13 for a feature-by-feature mapping.

CMS (SaaS)

CMS (SaaS) sets the indexing type per property outside of code. Use either the CMS UI or the content type REST API. The indexingType field accepts three values:

queryable – Filterable and sortable, but not full-text searchable.

searchable – Filterable, sortable, and full-text searchable.

disabled – Kept out of the Graph schema entirely.

Omit indexingType to get the retrieve-only behavior. The API accepts no default string value.

For details, see Define content typesDefine content types and Manage content types using the REST APIManage content types using the REST API.

If you have already reached the limit

If your Graph account has reached 100,000 fields and Graph is rejecting additional content types or properties, recover in two steps.

Reduce indexed fields where possible

Work through your content types and reduce indexing on every property you do not filter or search. Set each one to the retrieve-only behavior for your version. Use OutputOnly or OnlyStored in CMS 12, Default in CMS 13, or omit indexingType in CMS (SaaS). To drop a property from the schema entirely, use ExcludeField in CMS 12 or Disabled in CMS 13 and CMS (SaaS).

Run a smooth rebuild

Changing indexing settings does not remove existing over-indexed fields from your Graph account. To apply the updated schema and remove the fields you no longer need, run a smooth rebuild.

A smooth rebuild follows a blue-green deployment model. Graph creates a green slot alongside your live blue slot. It re-syncs your content into the green slot with the updated indexing settings and builds a clean schema. Query the green slot with the cg-query-new: true HTTP header to verify it before you promote it. When you are satisfied, click Accept to promote the green slot to live. If something is wrong, click Abandon.

Warning Until you accept or abandon the rebuild, Graph syncs content only to the green slot, not to the live slot. Content updates are not visible on the live site during this window. Schedule the rebuild for a low-traffic period, and avoid publishing content you need live until you accept or abandon it.

For the full process, see Smooth rebuildSmooth rebuild.

This page covers how Graph indexes the properties of a content item. For which content items Graph indexes in the first place, see What content Optimizely Graph indexes.

Version-specific references:

CMS 12 – Conventions APIConventions API

CMS 13 – Indexing conventionsIndexing conventions, Migrate the Conventions API from CMS 12 to CMS 13Migrate the Conventions API from CMS 12 to CMS 13

CMS (SaaS) – Define content typesDefine content types, Manage content types using the REST APIManage content types using the REST API