Support dictionary encoding for numeric types (DOUBLE/FLOAT/INT32/INT64) #1069

Summary

TsFile's DICTIONARY encoding is only wired for STRING/TEXT. Numeric types (DOUBLE/FLOAT/INT32/INT64) fall through to return nullptr in cpp/src/encoding/encoder_factory.h, so low-cardinality numeric data cannot use dictionary encoding.

Motivation

Parquet auto-detects low cardinality and dictionary-encodes float64 columns (RLE_DICTIONARY), storing indices instead of 8-byte values. This is the dominant reason Parquet compresses low-cardinality numeric data better than TsFile.

Benchmark on a 5214-column wide table of minute-level stock features (discrete/quantized values, ~1% distinct — e.g. 288k values with only ~2–4k distinct), all 1-chunk + zstd:

None of TsFile's existing numeric encodings match Parquet: they target smoothness/delta (Gorilla, TS_2DIFF, Chimp, Camel) or runs (RLE, RLBE), while this data is low-cardinality but scattered (no smoothness, no long runs).

Proposal

Extend DICTIONARY (or add a numeric dictionary encoding) to support DOUBLE/FLOAT/INT32/INT64. A numeric dictionary builds a map from each distinct value (bit pattern, or scaled integer) to a compact index, stores the dictionary once, then stores indices per value — the same mechanism Parquet uses.

Related

• Expand Java/C++ cross-language compatibility matrix to full coverage #1068 added FLOAT/DOUBLE RLE (run-length), but RLE only compresses consecutive repeats, not scattered low-cardinality repetition.