Skip to main content
By default, all non-text and non-JSON columns are indexed using ParadeDB’s columnar format. This enables fast filtering pushdown, Top K ordering, and aggregates over these columns. For example, in the following index definition, rating and id are columnar indexed because they are integers, whereas description is not because it is text.
To enable columnar indexing for text and JSON columns, cast the column to a tokenizer with columnar set to true.
The columnar option for tokenizers is available in versions 0.22.0 and above.
Columnar defaults to false for all tokenizers besides literal and literal normalized, which default to true and do not require an explicit setting. The reason is that tokenized columns can represent large documents and would be expensive to store column-wise, whereas literal and literal normalized columns are typically single-value and much more compact.
The columnar value stores the raw text value regardless of the tokenizer. For example, if Hello world is split into tokens hello and world, the columnar value remains Hello world.This is important because operations like filtering and sorting require the original column value, not the tokens.