About full-text search service

The Optimizely Full-Text Search Service (FTS Service) is a stand-alone WCF REST Service on top of an unmodified version of the open source search engine Lucene .NET and uses Atom with extensions described in Atom and Extensions as data protocol. The service is deployed through the EPiServer.Search Nuget package. Services are hosted by the IIS and are invoked by the IndexingService.svc` file, which means that multiple services can exists within the same environment. One service also can host multiple indexes as described in Named Indexes and Multi-Search. The following image shows a typical setup:

The EPiServer.Search.IndexingService assembly contains the following namespaces:

The Optimizely FTS Service defines the following REST endpoints:

When the ReferenceId extension contains a value, the Optimizely FTS Service determines that it is not a standalone item but an item that belongs to another item already in the index, such as comments in Optimizely Community. When a comment is added to the index, the ReferenceId is set to the commented entity, and when a query expression matches the contents of the comment, the commented entity returns as the IndexResponseItem and not the comment itself.

Any IndexRequestItem with IndexAction.Add and with a ReferenceId set, is added to the reference index automatically created for configured indexes with the suffix _ref. The search engine internally finds the parent item corresponding to the ReferenceId (by ID) and updates it so that all contents of any items in the _ref index for the parent item is added to the parent item searchable metadata. In this way, there is no need to send data over and over when updating the entity client side. You cannot search specific fields in reference data because default searchable fields are combined and added to the main item's metadata.

You cannot update the ReferenceId property for an item after it is added. Items with the IndexActionUpdate or Remove automatically updates its parent in the index if it was originally added with the reference ID set.

File content is indexed, the installed Ifilters decide which files are included.

When the DataUri extension contains a value, the Optimizely FTS Service immediately enqueues the whole request (in memory queue). Dequeueing of a DataUri request results in a call to the URI where the contents is concatenated to the IndexRequestItem.DisplayText and IndexRequestItem.MetaData, according to the configured value of maxDisplayTextLength.

The Optimizely FTS Service currently only supports File Uri’s and where the file needs to be accessible locally from the service perspective. You can override the behavior by overriding the GetFileUriContent(Uri) method or GetNonFileUriContent(Uri) in the IndexingServiceHandler.

The Optimizely FTS Service lets you configure multiple indexes, which you can use where there is an obvious separation of indexed content or where there is an existing (Lucene-compatible) index that is updated from a different source. Multiple named indexes need to be configured so that the index documents' fields map to the service's pre-defined field names. See Configure Optimizely full-text search service (Legacy)Configure Optimizely full-text search service (Legacy).

When you update the index, the target index (by name) is specified in the NamedIndex attribute extension of the Atom-formatted request. When you search the index, multiple indexes may be specified in the request, and the Optimizely FTS Service searches each one of the specified indexes and returns a merged result set. The default index is used if you do not specify any named index.

The VirtualPath feature enables structuring indexed content in a tree structure where you can search under a certain tree node. This is accomplished by storing the literal path together with the index document and searching the index with a path with a trailing wildcard. The path (when updating or searching) includes the full path from the root and up. For example: A document stored with the VirtualPath field set to node1/node2 would be a valid search result when you search for documents with the path node1 or node1/node2. However, it is not considered if the path only specified node2. If the IndexRequestItem.AutoUpdateVirtualPath attribute is set when you update or remove an item, search uses the VirtualPath to update or remove items under the provided node.

You can page search results at the service or at the client. Default configuration (useIndexingServicePaging=true) is service paging and returns only max numbers of items equal to the passed pageSize. Client paging returns max numbers of items equal to the configured maxHitsFromIndexingService(default 500). Consider the client paging option when filter providers are configured and paging needs to be intact.

The Optimizely FTS Service handles plain text only and does not understand a markup language, such as HTML, and cannot calculate relevance based on markup (such as <h1> <b> and so on).

The Optimizely FTS Service does not crawl the web or update automatically. Indexed content is pushed into the service and thus hands over the responsibility of updating the index to the client.

The loose couplings between the FTS Client and Service allow for third-party search engines to implement solutions compatible with the FTS Client independent of the platform. The only requirement is to comply with the REST service endpoint specifications. For .NET environments, override the UpdateIndex, GetSearchResults, GetNamedIndexes and ResetNamedIndex methods in the IndexingServiceHandler, thus using the existing WCF REST service implementation.