You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Preserve short IN lists for dictionary string columns #26945
Is your feature request related to a problem or challenge?
When Parquet RLE-to-dictionary conversion is enabled, the expression optimizer rewrites two- and three-value IN lists on dictionary string columns into equality chains. Each equality evaluates the dictionary values and maps its result through the keys separately. A single IN expression can evaluate membership on the dictionary values once and map one result through the keys.
TPC-H Q12 hits this with l_shipmode IN ('MAIL', 'SHIP') and o_orderpriority IN ('1-URGENT', '2-HIGH'). In a local main-versus-patch A/B with DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY=true, Q12 improved by 3.1-3.7% across focused and full-suite runs. Q13 and Q22, used as unchanged-plan controls, were flat.
Describe the solution you would like
Retain short literal IN and NOT IN expressions for dictionary columns whose values are Utf8 or Utf8View. Keep the existing rewrite behavior for non-dictionary string columns.
Describe alternatives you have considered
The dictionary physical expression could recognize and fuse equality chains, but preserving the original membership expression is smaller and avoids creating the repeated work in the first place.
Activity
Rich-T-kid commentedon Oct 30, 2026
take