Add search connectors
A search connector lets Optimizely Search & Navigation search content outside your website, such as an external blog related to your website. The connector lets your site visitors find the blog posts among your site's search results.
While Optimizely Search & Navigation instantly pushes internal content to the search engine, a connector crawls external websites on a specified schedule.
Configure connectors
To access the Connectors screen, select Search & Navigation > Configure > Connectors from the top menu. To view help, click Show Help in the top right corner. The following points supplement the help.
- Two connector types are available by default: Crawler and RSS/Atom. The connector type determines which configuration options display below the Type drop-down list.
- If you choose Crawler type, you can fine-tune indexing by excluding media types such as style sheets, JavaScript, and XML files. See also: Media Types. Optimizely Search & Navigation excludes the following media types from indexing by default.
- text/css
- text/ecmascript
- text/javascript
- application/ecmascript
- application/javascript
- application/x-javascript
- application/x-pointplus
Click Advanced fine-tuning of indexing to limit indexing further.
- Exclude query strings that are part of a link. For example, exclude crawling campaign tracking parameters (such as utm_source, used by Google Campaigns) to avoid unintentionally updating a campaign counter.
- Specify parts of a website to crawl but not index or crawl. You may want to crawl but not index search links to other pages, not the content on those pages.
- If you specify an indexing interval.
- Although you set a schedule in local time, it is converted to coordinated universal time (UTC), so it occurs simultaneously regardless of server location. However, you must manually adjust local time when needed, such as for daylight savings.
View connectors and indexing jobs
The connectors list (at the bottom of the screen) shows status and scheduling information for indexing jobs.
- You can manually refresh a connector's indexing status. If completed, the last completion time displays.
- You can edit or delete any connector from its context menu, and you can update its schedule, for example.
- You can manually start and stop indexing jobs.
Update a crawler's start URL
If you edit a connector's start URL, the crawl removes items previously indexed under the old URL from the index.