Elasticsearch types and index/query analyzers

You can select the data type for field boosting to handle the attribute or custom property. Most of these are situational, and the Default Data Type usually lets the system handle the indexing and searching of the data.

Treats the field as a binary True or False value.

Format: "true" or "false"

Elasticsearch Docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/boolean.html

Parses the field into a datetime that you can use to create date range searches.

Standard Format: "2015-01-01" or "2015/01/01 12:10:30".

Elasticsearch Docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/date.html

Use for structured content like an ID or emails. Useful for sorting and fields that have a set of values.

Elasticsearch Docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/keyword.html

Treat a field as numeric, which gives more relevant results when you use numbers in searches.

Elasticsearch Docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/number.html

Allows for full text searches and the use of an Index/Query analyzer for more flexibility in how the text in the field is parsed and searched.

Elasticsearch Docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/text.html

The Index and Query Analyzer options appear after you select Text as the Data Type for Field Boosting an attribute or custom property.

The analyzer list is a collection of Elasticsearch-compatible Analyzers. You should think about when and what data the analyzer is working against when selecting the analyzer.

The Index Analyzer processes against the data as it is added to the search collection. As a product is added to the index, it will go through the analyzer normalizing data on the product for the field tied to the Field Boosting record. The way it normalizes the field data is based on the Analyzer setup, as described below.

The Query Analyzer processes against the Search Query. As a frontend user types in a search, this query will be normalized against the data that was already Indexed.

Below is the list of standard platform analyzers. The Index and Query will be the same, but the data they are working against changes.

This is the standard Index Analyzer. If you have not configured anything, this is used to process a field during indexing. It will also remove HTML from the data.

Processing done on data: Standard/whitespace splitting, lowercasing, synonyms replacements, stop word removal, word stemming, and dimensional normalization.

Examples:

This is the same as the IscIndexAnalyzer but dooes not strip out HTML.

Processing done on data: Standard/whitespace splitting, lowercasing, synonyms replacements, stop word removal, word stemming, and dimensional normalization.

Examples:

This analyzer splits on whitespace and lowercases the characters.

Processing done on data: Whitespace lowercasing

Examples:

This analyzer performs standard lowercasing and removes non-alphanumeric characters.

Processing done on data: Standard lowercasing

Examples:

Ngram processing splits a word to help with partial matching during a search.

Ngram Analyzer changes the words of the field into partial tokens to trigger more relevant hits to documents/products/categories/content during a frontend search.

Processing done on data: Ngram tokenization and lowercasing. You can configure the Ngram settings to change the way the words are broken up.

Longer words will create more data, which increases the memory used by an index.

Examples:

Example: Min: 5 | Max: 8

'Hello, world! How are you DOING 2day?' -> ['hello', 'world', 'world!', 'orld!', 'doing', '2day?']

Example: Min: 3 | Max: 4

'United States of America USA United States Totally Different Text' -> ['uni', 'unit', 'unite', 'nit', 'nite', 'nited', 'ite', 'ited', 'ted', 'sta', 'stat', 'state', 'tat', 'tate', 'tates', 'ate', 'ates', 'tes', 'ame', 'amer', 'ameri', 'mer', 'meri', 'meric', 'eri', 'eric', 'erica', 'ric', 'rica', 'ica', 'usa', 'uni', 'unit', 'unite', 'nit', 'nite', 'nited', 'ite', 'ited', 'ted', 'sta', 'stat', 'state', 'tat', 'tate', 'tates', 'ate', 'ates', 'tes', 'tot', 'tota', 'total', 'ota', 'otal', 'otall', 'tal', 'tall', 'tally', 'all', 'ally', 'lly', 'dif', 'diff', 'diffe', 'iff', 'iffe', 'iffer', 'ffe', 'ffer', 'ffere', 'fer', 'fere', 'feren', 'ere', 'eren', 'erent', 'ren', 'rent', 'ent', 'tex', 'text', 'ext']

Example: Min: 5 | Max: 6

'United States of America USA United States Totally Different Text' -> ['unite', 'united', 'nited', 'state', 'states', 'tates', 'ameri', 'americ', 'meric', 'merica', 'erica', 'unite', 'united', 'nited', 'state', 'states', 'tates', 'total', 'totall', 'otall', 'otally', 'tally', 'diffe', 'differ', 'iffer', 'iffere', 'ffere', 'fferen', 'feren', 'ferent', 'erent']

The Dimensional Analyzer is very situational because it only normalizes specific terms to a specific form. This makes it easier to match since all values across a mapping will always be the same when found.

Processing done on data: Whitespace lowercasing and dimensional normalization. (Fractional normalization added in Elasticsearch version 7)

Examples: