rating and id are columnar indexed
because they are integers, whereas description is not because it is text.
columnar set to true.
The
columnar option for tokenizers is available in versions 0.22.0 and
above.false for all tokenizers besides literal and
literal normalized, which default to
true and do not require an explicit setting.
The reason is that tokenized columns can represent large documents and would be expensive to store column-wise,
whereas literal and literal normalized columns are typically single-value and much more compact.
The columnar value stores the raw text value regardless of the tokenizer. For example, if
Hello world is
split into tokens hello and world, the columnar value remains Hello world.This is important because operations like filtering and sorting require the original column value, not the tokens.